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Evaluating  the  Decision-Making  Skills  of  General  Aviation  Pilots 


This  report  describes  research  comparing  judg¬ 
ments  of  a  sample  of  general  aviation  pilots  (predomi¬ 
nately  Private  Pilot  certificate  holders)  and  an  expert 
panel  of  pilots  with  respect  to  decisions  critical  to 
flight  safety.  The  objective  was  to  develop  a  method¬ 
ology  to  assess  the  decision-making  skills  of  general 
aviation  pilots  by  using  a  written  format.  Current 
techniques  for  assessing  decision-making  in  pilots 
typically  rely  upon  the  observation  of  behavior  in 
either  a  flight  simulator  or  an  actual  aircraft  —  both 
of  which  are  both  time  consuming  and  expensive. 
Hence,  a  methodology  that  could  be  used  outside 
those  settings  would  have  significant  advantages  both 
for  research  and  for  training. 

L  BACKGROUND 

Based  on  a  review  of  36 1  general  aviation  accidents, 
a  National  Transportation  Safety  Board  report  (NTSB, 
1989)  concluded  that  97%  of  the  probable  causes 
were  attributable  to  flight  crew.  An  earlier  review  by 
Jensen  (1982)  attributed  as  many  as  85%  to  pilot 
error,  and  in  51.6%  of  the  fatal  accidents,  faulty 
decision-makings^^  cited  as  a  probable  cause. 

Studies  at  the  University  of  Illinois  (cf.,  Barnett, 
1989;  Stokes,  Kemper,  &  Marsh,  1992;  Wickens  et 
al.,  1987)  have  evaluated  pilot  decision  making  in  a 
simulated  environment  using  MIDIS  (Microcom¬ 
puter  based  Decision  Simulator).  These  studies  have 
generally  demonstrated  that  there  is  little  relationship 
between  pilot  demographic  characteristics  (such  as 
experience  levels)  and  performance  on  decision-mak¬ 
ing  tasks. 

Two  earlier  studies  (Driskill,  Weissmuller,  Quebe, 
Hand,  Dittmar,  &  Hunter,  1997;  Driskill, 
Weissmuller,  Quebe,  Hand,  &  Hunter,  1997)  exam¬ 
ined  pilot  decision-making  through  linear  modeling 
of  pilot  worth  functions.  Pilots  were  presented  with 
scenarios  in  which  weather  conditions  (ceiling,  vis¬ 
ibility,  and  amount  and  type  of  precipitation)  varied. 
Pilots  were  required  to  assign  “comfort  levels”  to  each 
scenario  that  indicated  their  relative  risk  assessments 
of  the  scenarios.  These  studies  indicated  that,  while 


pilots  tended  to  order  scenarios  involving  flight  under 
varying  favorable  and  unfavorable  weather  conditions 
(e.  g.,  unlimited  ceiling  and  visibility  and  no  precipi¬ 
tation  to  very  low  ceiling  and  visibility  and  freezing 
rain)  in  the  same  way,  there  were  wide  differences  in 
the  comfort  levels  they  assigned  to  flight  under  the 
varying  conditions.  These  results  imply  that  some 
pilots  either  do  not  know  the  safety  risk  implications 
of  flight,  or  some  are  more  willing  to  incur  safety  risks 
when  they  fly. 

The  studies  also  showed  that  expert  pilots  (sea¬ 
soned  instructors,  for  example)  had  consistent  ratings 
of  weather  conditions.  This  indicated  that  they  shared 
a  common  concept  of  what  constituted  “good”  and 
“bad”  weather  conditions.  In  the  weather  modeling 
studies,  this  common  concept  was  used  to  establish 
baseline  indices  for  the  generation  and  selection  of 
scenarios.  This  suggests  that  these  experienced  pilots 
may  also  share  common  concepts  of  “good”  and  “bad” 
decisions  in  other  aviation  settings,  and  that  those 
common  models  could  form  a  baseline  against  which 
the  decision-making  of  other  pilots  might  be  compared. 

The  present  study,  therefore,  sought  to  examine  the 
degree  to  which  the  decisions  of  less  experienced 
pilots  agreed  with  the  decisions  recommended  by 
expert  pilots  for  a  set  of  potentially  hazardous  in-fight 
events  presented  in  a  written  format.  Two  questions 
were  posed:  (1)  Do  the  judgments  of  less  experienced 
pilots  (about  500  hours  total  flying  hours)  differ  from 
those  of  an  expert  panel  in  making  decisions  critical  to 
flight  safety?  (2)  To  what  extent  are  demographic 
factors  associated  with  the  decisions  made?  Although 
previous  studies  (for  example,  Stokes,  Kemper,  & 
Marsh,  1992)  have  addressed  these  issues  to  some 
extent,  the  present  study  is  unique  in  the  use  of  a 
written  format  for  the  assessment  of  decision-making. 

In  the  research  described  below,  three  assumptions 
were  made:  (1)  The  affective  or  motivational  compo¬ 
nent  of  aeronautical  decision-making  (a  “Why  Fly” 
factor)  affects  pilots’  willingness  to  accept  higher 
safety  risk.  (2)  This  willingness  is  reflected  in  how 
they  judge  alternative  courses  of  actions  in  flights 
requiring  decisions  critical  to  flight  safety.  (3)  When 


1 


compared  with  expert  pilot  safety  risk  judgments, 
some  pilots  select  courses  of  action  having  greater 
safety  risk  than  other  actions  that  could  be  taken. 

A  flight  mission  or  “Why  Fly”  statement  was  used 
to  establish  the  motivational  dimension  in  each  sce¬ 
nario.  These  statements  were  based  on  the  following 
situational  factors  that  could  affect  the  way  pilots 
judged  the  courses  of  action  available  to  them: 

1 .  weather  phenomena 

2.  mechanical  malfunctions 

3.  biological  crises  (e.  g.,  sick  pilot  or  passenger) 

4.  social  influences  (e.  g. ,  passenger  requests  impacting 
flight  safety) 

5.  organizational  (e.  g.,  employer  or  Air  Traffic  Con¬ 
trol  requests  for  pilots  to  perform  some  action) 

The  remainder  of  this  report  presents  the  guidelines 
and  methods  for  developing  realistic  flight  descriptions 
and  details  about  capturing  expert  pilot  policy,  collect¬ 
ing  data  from  the  sample  of  general  aviation  pilots, 
computing  and  comparing  pilot  safety  risk  indices,  and 
analyzing  how  pilot  demographic  data  and  motivating 
conditions  relate  to  pilot  safety  indices. 

11.  SCENARIO  DEVELOPMENT 

Because  of  the  efficacy  of  scenarios  for  describing 
flights  simulating  situations  requiring  pilot  judgments 
(Driskill  et  al.,  1997),  this  study  employed  a  scenario- 
based  approach.  Each  scenario  would  consist  of  a 
succinct  description  of  the  situation  at  hand  and  four 
plausible  alternatives.  Although  some  studies 
(Wickens,  et  al,  1987;  Stokes,  Kemper,  &  Marsh, 
1 992)  have  suggested  that  the  use  of  a  multiple-choice 
format  attenuates  the  differences  between  novices  and 
experts,  the  overall  design  of  the  present  study  which 
relied  upon  self-administration  of  the  data  collection 
instrument  dictated  its  use.  How  scenarios  were  de¬ 
veloped  is  described  below. 

Guidelines 

Eight  principal  guidelines  directed  scenario  devel¬ 
opment: 

1 .  Content  of  each  scenario  should  be  based  on  situa¬ 
tions  that  have  actually  occurred  in  preparing  for  or 
making  a  flight. 

2.  Descriptions  should  clearly  require  a  decision  criti¬ 
cal  to  flight  safety. 


3.  Each  scenario  should  contain  four  plausible  and 
realistic  alternative  decision  choices  that  pilots  could 
reasonably  be  expected  to  make.  No  choice  should 
be  totally  unsafe;  nor  should  any  be  obvious  or  a 
“book  solution.” 

4.  Circumstances  or  events  representing  the  motiva¬ 
tional  conditions  (i.  e.,  weather,  mechanical,  bio¬ 
logical,  sociological,  and  organizational)  should  be 
either  explicit  or  stated  in  such  a  way  that  they  are 
self-evident  from  the  situation  presented. 

5.  Plausibility  and  realism  of  situational  descriptions 
and  associated  decision  choices  should  be  verified 
by  expert  pilots. 

6.  The  four  alternatives  should  contain  flight  safety 
risk  elements  differing  with  respect  to  amount  or 
degree  of  risk. 

7.  Scenario  content  should  eliminate  effects  of  local 
knowledge  on  the  part  of  respondents. 

8.  Because  the  data  collection  instrument  was  to  be 
administered  by  mail,  the  amount  of  time  required 
for  completion  should  not  exceed  one  hour. 

Sources  of  Scenario  Development 

The  task  scientist,  who  has  more  than  30  years 
experience  as  a  pilot,  based  development  of  the  sce¬ 
narios  on  four  sources. 

Previous  FAA  Research.  Specification  of  initial 
scenario  topics  was  based  on  the  results  of  a  large 
survey  of  American  pilots  (Hunter,  1995).  A  nation¬ 
wide  sample  of  pilots  provided  their  own  “hangar 
flying  stories”  about  dangerous  situations,  courses  of 
action,  judgments  made,  and  actions  taken.  Each 
anecdote  was  accompanied  by  lessons  learned.  The 
survey  produced  135  anecdotal  descriptions  of  lessons 
learned,  e.  g.,  “Don’t  let  others  do  your  preflight  for 
you;”  and  “Do  not  ever  let  your  passenger  talk  you 
into  doing  something  your  better  judgment  says  don’t 
do.”  Frequently,  there  was  more  than  one  lesson 
learned.  In  these  cases,  the  primary  and  secondary 
factors  were  identified  as  potential  targets  for  scenario 
development. 

National  Transportation  Safety  Board  Records.  A 
review  of  NTSB  summaries  of  accident  and  incident 
reports  was  used  to  narrow  the  list  of  lessons  learned. 
The  review  identified  accidents  or  incidents  in  which 
the  details  matched  items  on  the  lessons  learned  list  as 
well  as  being  representative  of  the  five  motivational 
conditions.  Later,  details  in  the  summaries  were  used 
to  develop  scenario  content. 
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“Why  Fly”  Survey.  Scenarios,  except  those  involving 
weather  phenomena  and  mechanical  malfunctions,  re¬ 
quired  a  clear  statement  of  flight  mission  or  motivation. 
To  determine  the  relative  value  of  motivational  state¬ 
ments  for  biological,  organizational,  and  sociological 
scenarios,  30  pilots  provided  ratings  of  25  of  the  most 
common  reasons  for  flying.  In  the  group  of  pilots,  10 
were  flight  instructors  and  16  were  airline  transport 
pilots.  Seventeen  of  them  were  employed  as  pilots.  They 
rated  each  reason  for  flying  based  on  their  willingness  to 
accept  risks  in  order  to  complete  a  flight,  using  a  scale  of 
0  (only  willing  to  accept  the  least  risk)  to  1 00  (willingness 
to  accept  greatest  risk).  They  also  provided  the  number 
of  times  they  had  flown  for  each  of  the  reasons  rated. 


Measures  (intraclass  correlations)  of  interrater  agree¬ 
ment  and  stability  of  responses  about  reasons  for  accept¬ 
ing  greater  risk  and  for  number  of  times  flown  for  each 
reason  were  computed  using  GRPREL  from  the  Com¬ 
prehensive  Occupational  Data  Analysis  Programs 
(CODAP).  This  analysis  package  is  briefly  described  in 
Appendix  E.  Interrater  agreement  (r^j)  was  .65  and  .49, 
respectively.  Stability  (tj^^)  of  responses  was  .98  and  .96. 
Stability  measures  above  .90  indicate  that  a  survey  of 
another  comparable  sample  of  subjects  would  yield 
highly  similar  results. 

Means  for  each  reason,  also  computed  by  GRPREL, 
are  shown  in  Table  1 .  Review  of  the  means  indicates  that 
the  reasons  pilots  might  be  willing  to  accept  greater  risk 


Table  1.  Mean  Ratings  of  Reasons  for  Taking  Greater 
Flight  Risks 


Reason 

Mean  Ratino 

Home-Holidays 

Hiah  Rated 

87.73* 

Search-Rescue 

79.70 

Angel  Flight 

74.97 

Deliver  Spare  Parts 

Midranae  Rated 

45.57 

Hunting  Trip 

41.37 

Instruct  Students 

38.63 

Sightseeing 

36.87 

Proficiency  Flight 

36.45 

Fly  Boss  to  Meeting 

35.43 

Resort-Vacation 

33.77 

Photograph-Show  Clients 

31.55 

Property 

Fly  Co-Workers  to  Meeting 

26.17 

Fly  Home  (After  3  Days) 

12.10 

Fly  Home  (After  7  Days) 

Low  Rated 

2.31 

Visit  Friends 

1.77 

Receive  Instruction-Flight 

1.72 

Check 

Fly  Home  (After  1  Day) 

1.47 

Friends  to  Ball  Game 

1.41 

Make  Airline  Connection 

1.40 

Home-Funeral 

1.10 

Fly  Self  to  Meeting 

1.07 

Administer  Flight  Checks 

0.87 

Deliver  Serum 

0.60 

Ferry-Deliver  Airplane 

0.40 

Ferry  VIPs 

0.30 

^Maximum  Rating  is  100.00;  minimum  rating  is  0.00. 
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are  easily  categorized  as  high,  midrange,  and  low. 
Three  reasons  are  in  the  high  rating  category,  with 
ratings  ranging  from  74.97  to  87.73;  10  have  midrange 
ratings  from  12.10  to  47.10;  and  12  have  low  ratings 
of0.30  to2.31. 

Subject  Matter  Expert  Interviews.  When  draft 
scenarios  and  alternatives  based  on  the  preceding 
sources  were  developed,  senior  pilots  completed  a 
field  review  of  candidate  scenarios.  These  senior  pilots 
provided  an  intensive,  critical  review  of  the  scenarios 
and  alternatives.  Participating  in  the  field  review  were 
present  and  former  FAA  Flight  Safety  Program  Man¬ 
agers  and  senior  flight  instructors  in  local  aviation 
schools.  On  the  basis  of  their  critical  reviews,  51 
scenarios  were  selected  for  use  in  the  study. 

Scenario  Content 

Aircraft.  Since  the  baseline  for  expert  pilot  risk 
assessments  was  a  pilot  with  approximately  500  flying 
hours,  the  most  commonly  flown  aircraft  used  by 
these  pilots  appeared  to  be  the  Cessna  172.  Flight  in 
a  rented  aircraft  of  this  type  was  the  basis  of  each 
scenario.  The  choice,  however,  restricted  the  me¬ 
chanical  and  performance  scenarios  to  fairly  basic 
malfunctions,  when  compared  with  some  of  the  air¬ 
craft  incidents  reviewed.  To  make  clear  to  all  respon¬ 
dents  (experts  and  national  sample),  a  flier  from  an 
airplane  rental  agency  was  included  in  the  survey.  The 
flier  described  the  airplane  and  onboard  equipment. 

Virtual  Airspace.  During  the  development  stage,  a 
decision  was  made  to  recast  scenarios  from  the  FAA 
survey  and  NTSB  summary  records  into  a  virtual 
airspace  designed  around  six  fictitious  airports  con¬ 
taining  the  characteristics  necessary  to  support  sce¬ 
nario  alternatives.  Several  factors  influenced  this 
decision.  First,  since  the  sources  of  the  scenarios 
involved  incidents  or  accidents  occurring  across  the 
United  States  and  several  foreign  nations,  mainte¬ 
nance  of  each  scenario  in  its  original  setting  would 
have  required  extensive  descriptions  and  possible  sup¬ 
porting  charts  and  airfield  diagrams.  Second,  the 
effects  of  “local  knowledge”  should  be  eliminated.  If 
actual  airfields  were  used,  it  is  possible  that  some 
respondents  might  be  familiar  with  them  and  would 
bring  some  level  of  terrain  or  airspace  “local  knowl¬ 
edge”  into  the  decision-making  process. 


Airport  diagrams,  patterned  upon  actual  airport 
directories,  were  constructed  to  support  the  scenarios. 
The  latitudes  and  longitudes  were  fictionalized  and 
set  in  the  middle  of  the  United  States  to  minimize 
references  to  similar  airports.  Included  along  with  the 
airfield  diagrams  was  information  about  runway  length 
and  lighting,  indicating  whether  the  tower  operated 
on  a  24-hour  schedule,  availability  of  airport  surveil¬ 
lance  radar,  and  listing  telephone  and  maintenance 
availability.  Because  such  information  is  not  normally 
available  to  private  pilots,  a  diagram  of  the  fictitious 
Air  Force  base  was  not  included.  To  discourage  re¬ 
spondents  from  forming  alternatives  not  in  the  four 
provided  for  each  scenario,  a  comprehensive  map 
showing  all  of  the  airfields  was  not  included. 

Scenario  Format.  Each  scenario  contained  a  stem 
describing  the  circumstances  of  the  flight  and  clearly 
showing  the  need  for  a  decision  critical  to  flight  safety. 
The  stem  for  biological,  sociological,  and  organiza¬ 
tional  scenarios  contained  an  explicit  statement  of  the 
mission  of  the  flight.  Results  of  the  “Why  Fly”  survey 
were  used  to  balance  motivating  conditions  used  in 
the  stem.  Motivation  levels  ranged  from  high  (>75)  to 
very  low  (<12)  risk  acceptance. 

The  need  for  a  critical  decision  was  self-evident  in 
mechanical  and  weather  scenarios,  since  these  kinds 
of  scenarios  do  not  require  a  direct  statement.  Pilots 
at  cruising  altitude  when  their  engine  quits,  for  ex¬ 
ample,  do  not  have  to  consider  the  purpose  of  their 
flight  in  order  to  initiate  their  decision-making  pro¬ 
cesses  to  resolve  their  problem. 

Each  scenario  stem  was  followed  by  four  alternative 
decision  choices.  Intensive  review  by  expert  pilots 
assured  that  each  alternative  was  plausible  —  that  is, 
each  offered  a  decision  a  500-hour  pilot  might  reason¬ 
ably  be  expected  to  make  —  and  differed  with  respect 
to  the  degree  of  safety  involved. 

Scenario  Structure 

The  final  set  of  scenarios  used  for  data  collection 
consisted  of  5 1  scenarios  with  all  supporting  informa¬ 
tion  (e.g.,  airport  diagrams).  By  situational  category, 
the  distribution  was  as  follows:  mechanical  8,  weather 
14,  biological  8,  sociological  10,  and  organizational 
1 1 .  Motivational  levels,  based  on  the  “Why  Fly” 
study,  were  distributed  as  follows:  high  or  strong,  2; 
middle  range,  14;  and  low  range  or  weak,  35. 
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IIL  CAPTURING  EXPERT  PILOT  POLICY 

Formatting  of  the  data  collection  instrument,  in¬ 
structions  for  performing  the  three  judgment  tasks, 
and  criteria  for  selecting  members  of  the  expert  panel 
are  described  below. 

Formatting  and  Instructions  for  Expert  Pilot 
Judgment  Tasks 

During  the  development  of  the  scenarios,  the  field 
review  by  subject  matter  experts  made  it  apparent  that 
capturing  the  judgments  of  three  factors  for  each  of 
four  alternatives  would  be  very  time  consuming  — 
possibly  requiring  two  or  more  hours.  Judgments  of 
30-35  scenarios  were  the  most  that  could  be  captured 
in  one  hour.  As  a  result,  the  5 1  scenarios  were  divided 
into  two  subsets  that  were  reproduced  as  General 
Aviation  Pilot  Study  Form  A  and  Form  B.  Each  form 
contained  33  scenarios.  Fifteen  scenarios  were  com¬ 
mon  to  both  Form  A  and  B;  the  remaining  18  sce¬ 
narios  were  unique  for  each  form. 

Instructions  were  provided  for  the  three  kinds  of 
rating  tasks  the  experts  were  asked  to  perform.  For  the 
first  task,  pilots  were  asked  to  rank  each  scenario 
alternative  with  respect  to  ‘‘...the  best  course  of  action 
fora  pilot  with  a  Private  Pilot  Certificate  (PPL)  who 
has  not  taken  the  Instrument  written  or  Flight  Check, 
and  has  approximately 500  total  hours  of  flying  time.  ” 
Alternatives  were  rated  a  i  for  the  best  course  of  action 
to  4  for  the  worst. 

In  the  second  rating  task  the  experts  were  asked  to 
rate  the  plausibility  of  each  alternative  in  terms  of  the 
percentage  of  500-hour  pilots  who  might  choose  the 
courses  of  action  described  in  each  scenario.  The 
experts  used  a  scale  ranging  from  0  to  100  percent. 

The  third  rating  task  asked  the  experts  to  estimate 
the  risk  associated  with  each  alternative.  Specifically, 
instructions  stated  enter  a  number  rangingfrom  0  to 
100  which  would  best  represent  your  estimate  of  how 
risky  this  course  of  action  would  be  for  a  PPL  with  500 
flying  hours  to  successfully  complete.  " 

A  page  illustrating  the  layout  of  the  scenarios  and 
alternatives  for  rating  by  the  safety  experts  is  provided 
in  Appendix  A. 

Safety  Expert  Criteria 

Several  criteria  were  established  for  selecting  pilots 
to  form  the  expert  safety  panel  for  assessing  their 
safety  risk  judgments. 


1.  Since  the  objective  of  this  research  required  expert 
panel  members  be  pilots  whose  focus  was  overtly  on 
flight  safety,  the  first  criterion  for  identifying  ex¬ 
perts  was  their  employment  in  positions  in  which 
flight  safety  is  a  foremost  job  objective.  A  second 
criterion  required prima  facie  evidence  of  the  pilot’s 
interest  in  safety  above  and  beyond  that  exhibited 
by  the  previous  national  samples  of  pilots.  Pilots 
meeting  these  criteria  included  FAA  staff  members, 
senior  instructional  staff  in  university  aviation  pro¬ 
grams,  and  pilots  who  frequently  attended  safety 
seminars. 

2.  As  a  group,  the  level  of  experience  should  provide 
them  with  knowledge  of  the  situations  described  in 
the  51  scenarios.  In  addition  to  age,  21  other  expe¬ 
rience  items  were  developed,  ranging  from  total 
hours  logged,  certification  and  ratings,  to  total  and 
recent  night  and  weather  flying  experience. 

3.  The  expert  responses  were  carefully  examined  for 
interrater  agreement  (r^^)  and  stability  (r^^  and 
assessed  for  the  presence  of  multiple  policies  with 
regard  to  the  rank  ordering  of  the  scenarios  and 
ratings  of  percentage  of  safety  risk.  The  minimum 
r^j  was  .20  which  represents  the  average  correlation 
of  each  rater  with  each  other  rater.  The  minimum 
for  was  .90;  values  at  this  level  or  higher  indicate 
high  probability  that  another  set  of  raters  would 
provide  comparable  ratings.  In  addition,  GRPREL 
assesses  each  rater  in  terms  of  agreement  with  other 
raters  and  identifies  “deviant”  raters.  Deviant  raters 
are  those  whose  rating  policy  does  not  significantly 
correlate  (p  <  0.05)  with  other  raters.  This  process 
evaluates  the  existence  of  multiple  policies.  Usually, 
only  one  to  three  raters  are  identified  as  deviant 
because  they  did  not  follow  instructions  or  reversed 
the  scale  used.  When  there  is  a  larger  number  of 
deviant  raters,  GRPREL  assesses  them  in  the  same 
way  to  determine  if  there  is  a  significant  secondary 
policy.  A  description  of  the  GRPREL  analysis  sys¬ 
tem  and  representative  references  are  available  in 
Appendix  E. 

4.  The  experience  level  should  exceed  that  of  the  na¬ 
tional  population  in  terms  of  total  hours  and  levels 
of  certification. 

Collection  of  Expert  Data 

Forms  A  and  B  of  the  General  Aviation  Pilot 
booklet  were  personally  delivered  to  pilots.  They 
returned  their  responses  by  mail.  Judgments  were 
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obtained  from  3 1  pilots.  This  group  was  comprised  of 
FAA  staff  personnel,  including  the  present  and  former 
managers  of  the  Flight  Standards  District  Office 
(FSDO)  in  San  Antonio;  flight  instructor  personnel  at 
Embry-Riddle  Aeronautical  University,  University  of 
North  Dakota,  Daniel  Webster  College,  and  Ohio 
State  University;  and  pilots  attending  a  one-day  FAA 
sponsored  safety  seminar  in  San  Antonio,  Texas. 

Demographics  and  experience  of  the  pilots  com¬ 
prising  the  safety  expert  panel  compare  favorably  with 
the  criteria  set  for  panel  membership  and  are  reported 
in  Table  6  and  compared  with  the  sample  of  general 
aviation  pilots. 

Expert  Interrater  Agreement  and  Policy  Stability 

The  rank  order,  percentage  of  safety  risk,  and 
alternative  plausibility  judgments  of  31  experts  were 
entered  for  GRPREL  analysis. 

Rank  Order^  Interrater  agreement  (R^^)  and  stabil¬ 
ity  (Rjj^)  of  the  expert  rankings  of  alternative  courses 
of  action  are  .44  and  .93,  respectively.  Both  values 
exceed  the  criteria  set  for  agreement  and  stability. 
There  was  no  evidence  of  a  secondary  policy. 

Safety  Risk^  Interrater  agreement  (Rji)  and  stabil¬ 
ity  (Rjj^)  of  the  expert  rankings  of  alternative  courses 
of  action  are  .45  and  .94,  respectively.  These  values 
are  almost  identical  to  those  for  rank  order. 

The  distribution  of  mean  ratings  is  shown  in  Table 
2.  Percentage  of  safety  risk  means  for  the  alternatives 
are  distributed  from  2.08  (lowest  risk)  to  86.95  (high¬ 
est  risk).  The  grand  mean  of  the  rankings  is  36.52, 
with  a  standard  deviation  of  19.19. 

Alternative  Plausibility.  In  making  their  plausibil¬ 
ity  ratings,  the  experts  were  asked  to  estimate  the 
percentage  of  500-hour  pilots  who  might  choose  the 
course  of  action  represented  in  each  alternative.  The 


Rjj  and  for  these  ratings  were  .22  and  .84,  respec¬ 
tively,  based  on  30  raters.  The  grand  mean  of  the 
plausibility  ratings  for  the  204  alternatives  was  37.87 
with  a  range  from  9.09  (lowest  plausibility)  to  76. 1 1 . 
The  distribution  of  ratings  is  shown  in  Table  3. 
Overall,  the  ratings  appeared  to  be  reasonable  and 
plausible. 

Effect  of  “Why  Fly”  on  Expert  Ranking  and 
Rating 

The  ‘'Why  Fly”  statement  in  each  scenario  was 
intended  to  represent  the  motivational  component  of 
aeronautical  decision-making.  If  the  statement  had  its 
intended  effect,  first-choice  alternatives  for  scenarios 
should  be  expected  to  have  higher  risk  ratings  than 
alternative  courses  of  action  —  that  is,  the  first- 
ranked  alternative  would  not  be  the  alternative  with 
the  lowest-rated  safety  risk.  A  Pearson  product  mo¬ 
ment  correlation  of  alternatives  rankings  and  ratings, 
however,  was  .808,  indicating  that  the  “Why  Fly” 
statements  had  little  overall  effect  on  the  rankings  and 
ratings.  Given  this  high  correlation,  any  “Why  Fly” 
effect  is  most  likely  to  be  specific  to  a  given  flight 
scenario. 

The  relationship  between  rankings  and  ratings  is 
depicted  in  Table  4.  This  table  shows  the  number  of 
times  the  first-ranked  alternative  for  a  scenario  corre¬ 
sponded  with  the  alternative  with  the  lowest  safety  risk 
rating.  Note  that  the  risk  ratings  for  each  alternative  were 
converted  to  rank  order  to  make  the  relationships  more 
easily  perceived.  Several  factors  should  be  noted  in  the 
Table  4  data.  First,  for  42  of  the  scenarios,  the  first- 
ranked  choice  also  had  the  lowest  risk.  Only  nine  sce¬ 
narios  had  a  higher  risk  alternative  than  the  first-ranked 
alternative.  Of  these  nine,  the  first-ranked  choice  had  the 
second  lowest  risk  percentage  for  four  scenarios.  The 


Table  2.  Distribution  of  Safety  Risk 
Ratings  Across  the  204  Alternatives 
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Table  3-  Distribution  of  Plausibility 
Ratings  Across  the  204  Alternatives 
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Table  4-  Comparison  of  Expert 
First  Choices  with  Risk  Ratings  for 


51  Scenarios 

Rank 

Risk 

N  Exoerts 

Order 

Order’ 

1 

1 

42 

1 

2 

4 

1 

3 

5 

2 

1 

6 

2 

2 

35 

2 

3 

8 

2 

4 

2 

3 

1 

3 

3 

2 

9 

3 

3 

29 

3 

4 

10 

4 

2 

3 

4 

3 

9 

4 

4 

39 

’Percentage  of  risk  converted  to  rank 

order;  1  =  lowest  risk,  4  = 

highest  risk 

remaining  five  had  the  next  to  highest  risk  alternative  as 
the  first  ranked  choice.  No  first-ranked  choice,  however, 
had  the  highest  risk  percentage. 

Second,  the  highest  coincidence  of  rank  and  risk 
rating  was  for  the  first-  and  fourth-ranked  choices.  As 
indicated,  for  42  scenarios  the  first-ranked  alternative 
also  had  the  lowest  percentage  of  risk.  For  39  sce¬ 
narios,  the  fourth-ranked  choice  also  had  the  highest 
risk  percentage. 


Third,  agreement  between  rank  order  and  associ¬ 
ated  risk  percentage  was  less  for  the  second  and  third 
ranked  alternatives.  Rank  ordering  and  percentage  of 
risk  coincided  for  35  and  29  scenarios,  respectively. 

Scenario-specific  information  for  the  nine  scenarios 
where  the  first-ranked  alternative  was  not  the  alterna¬ 
tive  with  the  lowest  safety  risk  rating  is  in  Table  5.  For 
six  of  the  scenarios,  the  differences  of  percent  safety 
risk  were  small,  ranging  from  .70  to  8.42. 

Risk  percentages  were  higher  for  three  other  sce¬ 
narios  for  which  the  “Why  Fly”  statements  may  have 
influenced  the  rankings.  Scenario  36  describes  flight 
to  deliver  medical  serum  to  an  American  Red  Cross 
team,  and  a  potential  mechanical  problem  is  reported 
to  the  pilot.  Scenario  46  involves  flight  to  a  football 
game  where  a  weather  problem  intervenes  in  the 
flight.  Scenario  51  describes  a  sight  seeing  trip  with 
friends  when  a  potential  weather  problem  develops. 

IV.  COLLECTING  GENERAL  AVIATION 
PILOT  JUDGMENTS 

Data  Collection  Instrument 

The  5 1  scenarios  were  reproduced  in  random  order 
in  a  booklet  entitled  “General  Aviation  Pilot  Study” 
(see  Appendix  B).  To  reduce  the  amount  of  time 
required  to  complete  the  research  task,  the  pilots  were 
asked  only  to  rank  order  the  alternative  courses  of 
action  for  each  scenario.  The  study  booklet  consisted 
of  a  background  information  section,  instructions  for 
ranking  scenario  alternatives,  a  pamphlet  describing 
the  aircraft,  airport  diagrams  and  associated  informa¬ 
tion,  and  the  scenarios. 


Table  5.  Summary  of  Expert  Rankings  and  Safety  Risk  Ratings 
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The  background  information  section,  titled  Pilot 
Information,  elicited  the  same  demographic  and  ex¬ 
perience  information  from  the  national  sample  as  was 
obtained  from  the  expert  panel  of  pilots.  In  addition, 
the  sample  pilots  were  asked  to  provide  information 
about  the  aircraft  they  flew  most  frequently. 

The  instructions,  after  explanation  of  the  informa¬ 
tion  supporting  the  scenarios,  carefully  described  how 
alternatives  should  be  ranked,  the  substance  of  which 
is  quoted  below: 

First,  carefully  read  the  scenario  and  the  four  listed 
alternative  responses.  Assume  you  have  leased  the  Cessna 
172  shown  on  the  flyer  from  Aircraft  Rental  and 
Leasing.  Feel  free  to  use  the  airfield  diagrams  for  assis¬ 
tance  in  understanding  the  problem. 

Second,  based  on  your  experience,  decide  which  of 
the  alternatives  you  would  most  likely  select  as  your  first 
course  of  action,  what  would  be  your  second  choice, 
then  your  third  and  fourth  if  you  were  the  pilot  in  the 
scenario.  To  the  left  of  each  choice  are  the  numbers  (1 
234),  circle  the  1  next  to  the  choice  you  selected  as  your 
first.  Next  select  the  choice  you  consider  second  and 
circle  the  2  to  the  left  of  the  choice,  then  circle  your  third 
and  fourth  choices. 

An  example  showing  how  rankings  were  to  be 
marked  was  included. 

The  instructions  explicitly  stated  that  “This  is  not  a 
test.”  Further,  they  stated  “There  are  no  right  or  wrong 
answers,  and  you  are  not  asked  to  do  anything  illegal.”* 

Data  Collection  and  General  Aviation  Sample 
Demographics  and  Experience 

Data  Collection.  The  data  collection  instrument 
was  mailed  to  approximately  1 ,000  private  pilots  in 
the  Eastern,  Southwest,  and  Northwest  Mountain 
Regions  of  the  FAA.  Pilots  were  randomly  selected 
from  current  FAA  records.  The  package  contained  a 
postage-paid  envelope  addressed  to  the  Office  of  Avia¬ 
tion  Medicine  in  Washington,  D.  C.,  for  pilots  to  use 
in  returning  their  responses.  Responses  from  246 
pilots  were  returned  to  the  Office  of  Aviation  Medi¬ 
cine  and  key  entered  for  data  analysis. 


Sample  Demographics  and  Experience.  Sample 
demographics  and  flying  experience  are  compared 
with  similar  information  for  the  3 1  pilots  in  the  expert 
safety  panel  in  Table  6.  A  complete  summary  of  the 
demographic  and  flying  experience  of  the  general 
aviation  sample  is  provided  in  Appendix  C. 

The  sample  statistics  support  two  conclusions.  First, 
the  sample  satisfied  the  objective  of  the  study  to 
obtain  the  judgments  of  pilots  with  about  500  total 
flying  hours.  In  addition,  most  of  them  (70%)  were 
not  instrument-rated.  Second,  the  criterion  related  to 
the  experience  requirements  of  the  expert  panel  of 
pilots  was  achieved,  as  the  pilots  in  the  GA  sample  had 
considerably  less  experience  that  the  experts,  nor  did 
any  of  the  general  aviation  pilots  hold  CFI  or  CFII 
certificates. 

The  GA  pilot  sample  is  comprised  mostly  of  pilots 
from  Texas  (129),  New  York  (63),  and  Washington 
(34),  although  pilots  from  12  states  are  represented. 
The  most  distinguishing  features  of  the  national  sample 
are  the  ownership  of  their  own  aircraft  (by  50%  of  the 
pilots)  and  the  less  than  750  hours  of  total  flying  hours 
(by  76%  of  them).  Other  flying  experience,  as  would 
be  expected,  is  similarly  limited:  78%  have  fewer  than 
50  hours  night  flying  and  over  one-half  have  fewer 
than  24  hours  of  night  flying.  The  aircraft  they  most 
frequently  fly  is  equipped  with  a  transponder  (96%), 
75%  are  IFR-capable,  37%  are  autopilot  equipped, 
but  only  nine  percent  are  equipped  with  weather 
radar.  Most  of  the  pilots  (93%)  report  flying  single¬ 
engine  aircraft.  While  all  are  single-engine  rated, 
only  11%  are  multi-engine  rated;  none  are  certi¬ 
fied  as  CFI. 

V.  ANALYSIS  OF  SCENARIO  ITEM 
RESPONSES 

The  mean  vectors  of  rankings  for  the  general  avia¬ 
tion  sample  and  the  expert  panel  of  pilots  were  corre¬ 
lated,  and  a  Pearson  r  of  0.914  was  obtained.  The 
general  aviation  sample  and  experts  rank-ordered  the 
alternatives  differently  for  only  seven  scenarios.  Table 
7  lists  the  seven  scenarios  where  the  differences 
occurred,  along  with  the  first  choice  alternative  of  the 


*  After  virtually  all  the  data  collection  had  been  completed  it  was  discovered  that  some  of  the  scenarios  depicted  VFR  flights  at  altitudes 
normally  used  for  IFR  flight.  According  to  the  Airman’s  Information  Manual,  VFR  flights  in  an  easterly  direction  should  be  conducted 
at  an  odd  thousand  feet  plus  500  feet  (for  example,  7,500  feet);  while  westerly  VFR  flights  should  be  conducted  at  an  even  thousand 
feet  plus  500  feet  (for  example,  6,500  feet).  However,  some  of  the  scenarios  used  in  the  data  collection  indicated  that  the  pilot  was 
flying  VFR  at  a  whole  number  of  thousands  of  feet.  For  example,  the  second  scenario  reads,  “You  are  solo  on  a  late  night  cross  country 
cruising  VFR  at  9000  feet...”  While  this  is  not  illegal,  it  is  contrary  to  recommended  operating  procedures.  Neither  the  experts  nor 
the  general  aviation  pilots  commented  upon  this  error,  so  it  is  believed  not  to  have  significantly  influenced  the  results. 
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Table  6.  General  Aviation  Sample  and 
Expert  Panel  Demographics  and  Flying 
Experience  Comparison 


Demographic 

Sample 

Expert 

Mean  age  in  years 

46.9 

45.9 

Male 

96% 

90% 

Female 

4% 

10% 

Military  flying  experience 

3% 

23% 

Mean  CFI/CFII  years 

0.0 

12.0 

Percent  Certificates  Held 

Private  pilot 

98% 

6% 

Commercial 

2% 

35% 

ATP 

0% 

48% 

CFI 

0% 

67% 

CFII 

0% 

65% 

Instrument 

30% 

97% 

Single  engine 

100% 

93% 

Multi  engine 

11% 

90% 

Mean  fivina  hour  loaaed 

Total 

589 

4995 

Weather  (IMC)  flying  hours 

41 

581 

Instructor  flying  hours 

0 

1683 

Night  flying  hours 

43 

528 

Mean  fivina  hours  last  60  davs 

Total 

10 

55 

Day 

10 

50 

Weather 

1 

7 

Instructor 

0 

33 

Night 

2 

12 

experts  and  the  general  aviation  sample,  the  safety 
ratings  associated  with  each  alternative,  and  the  dif¬ 
ference  between  the  experts  and  sample. 

Item  Analysis  and  Score  Generation 

A  scoring  key  was  developed  by  taking  the  alterna¬ 
tive  ranked  #1  by  the  expert  panel  as  the  keyed 
alternative.  This  answer  key  is  given  in  Table  8.  Using 
this  key,  the  responses  of  the  general  aviation  sample 
were  analyzed  using  the  ITEMAN  Item  and  Test 
Analysis  Program  (Version  3.50,  Assessment  Systems 
Corp.).  This  program  generated  item  statistics  (given 
in  Table  9)  for  each  of  the  51  scenario-items. 

ITEMAN  also  generated  number-right  scores 
(Safety  Deviation  Index,  SDI)  for  all  subjects  and 
produced  overall  scale  statistics  for  the  SDI.  These 
statistics  are  given  in  Table  10.  The  SDI  scores  were 
merged  with  demographic  information  and  were  sub¬ 
sequently  analyzed  using  SPSS  for  Windows  (Ver 
6.0).  Figure  1  shows  the  distribution  of  SDI  for  the 
246  subjects.  As  may  be  seen  from  that  figure,  the 
distribution  of  scores  is  approximately  normal,  with  a 
mean  (and  median)  of  27. 

Analysis  of  sample  demographic  variables 

Correlations  were  computed  between  the  SDI  and 
the  sample  demographic  variables,  as  shown  in  Table 
1 1 .  Although  the  correlations  are  uniformly  small  and 
generally  not  statistically  significant,  they  form  an 
interesting  pattern.  Of  particular  note,  the  correla¬ 
tions  for  all  the  flying  experience  variables  have  a 
negative  sign,  while  the  single  correlation  with  a 
positive  sign  is  between  the  SDI  and  age. 


Table  7.  Mean  Rank  and  Percentage  of  Safety  Risk 
Differences  Between  Experts  and  Sample  Pilots 


Scenario 

1**-ranked 

Expert 

Sample  Expert  Sample 

Difference 

35 

b 

c 

Rating  Rating 

2.25  45.83 

43.58 

04 

c 

b 

30.71  45.00 

14.29 

48 

b 

c 

16.45  30.45 

14.00 

15 

c 

b 

8.33  15.20 

6.87 

39 

b 

a 

25.45  26.36 

0.91 

49 

c 

b 

32.73  33.64 

0.91 

51 

d 

c 

39.54  40.00 

0.46 

Note:  Difference  =  Sample  Rating  - 

Expert  Rating 
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Table  8.  Scenario  Keyed  Responses 


Scenario 

Keyed 

Scenario 

Keyed 

Scenario 

Keyed 

No. 

Response 

No. 

Response 

No. 

Response 

1 

b 

18 

a 

35 

b 

2 

b 

19 

d 

36 

d 

3 

d 

20 

d 

37 

b 

4 

c 

21 

b 

38 

b 

5 

d 

22 

d 

39 

b 

6 

b 

23 

b 

40 

d 

7 

c 

24 

b 

41 

b 

8 

a 

25 

b 

42 

b 

9 

d 

26 

d 

43 

c 

10 

d 

27 

c 

44 

c 

11 

c 

28 

b 

45 

b 

12 

c 

29 

c 

46 

b 

13 

d 

30 

a 

47 

d 

14 

b 

31 

d 

48 

b 

15 

c 

32 

b 

49 

c 

16 

a 

33 

a 

50 

d 

17 

b 

34 

d 

51 

d 

Note:  The 

keyed  response  is  that  which  was  rank-ordered  #1  by  the  expert  panel. 

Table  9.  Item  Statistics 


Item 

No. 

P 

D 

rpb 

Item 

No. 

P 

D 

Tpb 

Item 

No. 

P 

D 

rpb 

1 

.83 

.28 

.33 

18 

.89 

.24 

.35 

35 

.39 

.58 

.50 

2 

.70 

.47 

.40 

19 

.16 

.11 

.17 

36 

.08 

.04 

.05 

3 

.54 

.53 

.37 

20 

.88 

-.08 

-.07 

37 

.37 

.60 

.52 

4 

.44 

.08 

.07 

21 

.78 

.40 

.40 

38 

.44 

.37 

.32 

5 

.13 

.18 

.22 

22 

.94 

.08 

.20 

39 

.26 

.18 

.14 

6 

.24 

.23 

.22 

23 

.83 

.32 

.37 

40 

.51 

.20 

.21 

7 

.69 

.13 

.10 

24 

.22 

.27 

.33 

41 

.34 

.49 

.42 

8 

.57 

.48 

.43 

25 

.52 

.41 

.37 

42 

.53 

.17 

.13 

9 

.56 

.40 

.38 

26 

.51 

.29 

.24 

43 

.90 

.19 

.33 

10 

.16 

.20 

.27 

27 

.57 

.57 

.48 

44 

.97 

.06 

.19 

11 

.67 

.44 

.45 

28 

.61 

.22 

.18 

45 

.43 

.42 

.36 

12 

.71 

.46 

.44 

29 

.54 

.19 

.15 

46 

.28 

.45 

.44 

13 

.58 

.60 

.50 

30 

.57 

.09 

.07 

47 

.02 

.00 

-.01 

14 

.87 

.18 

.21 

31 

.46 

.40 

.35 

48 

.28 

-.12 

-.03 

15 

.44 

.28 

.30 

32 

.84 

.10 

.13 

49 

.27 

.06 

.07 

16 

.80 

-.02 

.00 

33 

.55 

.32 

.27 

50 

.85 

.28 

.37 

17 

.93 

.09 

.19 

34 

.17 

.35 

.42 

51 

.39 

.47 

.42 

P=  Proportion  Correct 
D=  Discrimination  Index 

rpb  =  point-biserial  correlation  for  keyed  response 
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Table  10.  Safety  Deviation  Table  11.  Correlations  of  the  SDI  and 

Index  statistics  demographic  variables 


N  of  Items 

51 

Variable 

r 

P 

N  of  Examinees 

246 

Age 

.1044 

.105 

Mean 

27.2 

Total  flying  hours 

-.0796 

.218 

Variance 

36.0 

Total  weather  flying  hours 

-.0991 

.151 

S.D. 

6.0 

Total  instrument  flying  hours 

-.1198 

.071 

Median 

27 

Total  night  flying  hours 

-.0306 

.642 

Alpha 

0.753 

Flying  hours  -  last  60  days 

-.1121 

.086 

Mean  P 

0.534 

Weather  flying  hours  -  last  60  days 

.-1417 

.033 

Mean  Biserial 

0.371 

Instrument  flying  hours  -  last  60  days 

-.0628 

.345 

Minimum  Score 

6 

Night  flying  hours  -  last  60  days 

-.1509 

.022 

Maximum  Score 

44 

Number  of  Pilots 


6  8  10  12  14  16  18  20  22  24  26  28  30  32  34  36  38  40  42  44 


SDI  Score 

Figure  1.  Distribution  of  Safety  Deviation  Index  scores. 


11 


VIL  CONCLUSIONS  AND  DISCUSSION 

This  study  sought  to  examine  the  decision-making 
skills  of  typical  general  aviation  pilots  by  comparing 
their  preferred  solutions  to  a  number  of  scenarios  with 
the  solutions  recommended  by  a  panel  of  experts.  It 
was  found  that,  overall,  general  aviation  and  expert 
pilots  agreed  in  their  judgments  of  the  appropriate 
course  of  action  in  situations  critical  to  flight  safety. 
That  is,  there  was  a  high  correlation  between  the 
vector  of  mean  recommended  solutions  provided  by 
the  expert  and  the  vector  of  mean  solutions  chosen  by 
the  general  aviation  pilots. 

Notwithstanding  the  high  overall  agreement  be¬ 
tween  general  aviation  and  expert  pilots,  the  degree  of 
agreement  by  individual  general  aviation  pilots  with 
the  recommended  solutions  varied  widely.  This  was 
evidenced  by  the  range  of  scores  (6  to  44)  obtained 
when  the  scenario  items  were  analyzed  using  a  simple 
right- wrong  scoring  system.  That  analysis  showed 
that,  on  average,  the  general  aviation  pilots  selected 
the  recommended  alternative  for  only  about  half  of 
the  scenarios. 

Two  demographic  factors  seem  to  influence  the 
general  aviation  pilots’  judgments:  age  and  quantity 
of  flying  experience.  Younger  pilots  tend  to  make 
judgments  involving  higher  risk  than  older  pilots. 
The  same  is  true  for  more  experienced  pilots  in  terms 
of  total  and  recent  flying  experience.  Because  the 
correlations  are  small  and  not  all  statistically  signifi¬ 
cant,  we  must  be  cautious  in  interpreting  these  results, 
as  the  trend  we  see  may  prove,  on  further  investiga¬ 
tion,  to  simply  be  a  spurious  statistical  result.  How¬ 
ever,  because  the  implications  are  intriguing,  and  with 
the  preceeding  caveat  in  mind,  we  will  explore  them 
briefly.  Recall  that  the  data  presented  earlier  (Table  4) 
suggest  that  the  expert  panel,  in  their  rank  ordering  of 
the  alternatives,  generally  assigned  the  #1  rank  to  the 
safest  (that  is,  least  risky)  of  the  four  alternatives 
given.  Higher  SDI  scores  are  therefore  indicative  of 
subjects  who  endorse  the  least-risky  (most  conserva¬ 
tive)  solution  for  each  scenario. 

The  data  suggest,  however  tentatively,  that  while 
age  is  positively  associated  with  increased  conserva¬ 
tism  (less  risky),  flying  experience  is  negatively  associ¬ 
ated  with  conservatism.  That  is,  older  pilots  are  more 
conservative  than  younger  pilots,  but  pilots  with  more 
experience  are  less  conservative  than  pilots  with  little 
experience.  Therefore,  the  pilots  who  are  the  least 


risk-averse  would  be  those  who  are  both  young  and 
who  have  higher  levels  of  total  and  recent  experience. 
This  interpretation  is  consistent  with  the  findings  of 
two  studies  of  weather  information  utilization 
(Driskill,  Weissmuller,  Quebe,  Hand,  Dittmar,  et  al., 
1997;  Driskill,  Weissmuller,  Quebe,  Hand,  et  al., 
1997)  in  which  pilots  rated  a  number  of  combinations 
of  weather  conditions  in  terms  of  their  comfort  level  in 
flying  under  such  conditions.  Hierarchical  cluster  analy¬ 
ses  indicated  that  the  groups  with  the  highest  comfort 
level  (that  is,  least  risk-averse)  were  the  youngest. 

Similar  results  were  also  obtained  from  a  study  of 
risk  factors  for  aircraft  accidents  in  New  Zealand 
(O’Hare,  Chalmers,  &  Bagnall,  1996),  in  which  a 
group  of  “high  flyer”  young,  active  pilots  was  over¬ 
represented  in  accidents.  However,  Stokes,  Kemper, 
&  Marsh  (1992,  pg  34)  investigated  decision-making 
in  simulated  flights  in  the  MIDIS  and  found  “...no 
evidence  that  better  decision  making  automatically 
comes  with  advancing  years  or  the  accumulation  of 
flight  hours.”  Since  the  methods  and  criteria  used 
among  these  studies  and  the  many  others  (cf.,  Barnett, 
1989;  Jensen,  1995;  Wickens  et  al.,  1987)  that  have 
evaluated  the  impact  of  age  and  experience  on  pilot 
performance  differ  considerably  from  those  used  in 
the  present  study,  it  is  difficult  to  assess  the  compara¬ 
bility  of  their  results.  Further,  we  must  again  note  that 
the  correlations  obtained  here  were  quite  small  indeed 
and,  for  the  most  part,  represent  only  trends  that  may 
not  prove  to  be  statistically  reliable.  Thus,  this  tenta¬ 
tive  finding  must  be  treated  with  considerable  caution 
but  may  be  worth  exploring  in  future  studies. 

The  main  purpose  of  the  study  seems  to  have  been 
accomplished.  That  is,  a  written  instrument  was  de¬ 
veloped  that  may  be  used  to  evaluate  the  decision¬ 
making  skills  of  general  aviation  pilots.  The  SDI 
derived  from  the  instrument  has  adequate  reliability 
as  measured  by  coefficient  alpha,  and  an  approxi¬ 
mately  normal  distribution,  centered  around  a  mean 
performance  level  of  approximately  50%  correct.  These 
are  necessary  (though  certainly  not  sufficient)  condi¬ 
tions  for  an  instrument  that  might  be  used  to  assess 
general  aviation  pilot  decision-making.  Additional 
studies  will  be  required  to  evaluate  the  relationship  of 
the  SDI  to  external  criteria  such  as  accident  involve¬ 
ment  or  involvement  in  potentially  hazardous  events 
(for  example,  running  low  on  fuel),  along  with  addi¬ 
tional  psychometric  properties  of  the  SDI  and  other 
potential  indices  derived  from  the  instrument. 
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Finally,  we  must  note  that  although  general  avia¬ 
tion  pilots  may  demonstrate  on  paper  that  they  have 
the  knowledge  and  perspective  for  deciding  upon  and 
taking  the  safest  course  of  action,  there  is  no  assurance 
that  in  real-time  situations,  under  the  pressures  and 
motivations  of  the  moment,  that  they  will  in  fact 
apply  this  knowledge  appropriately.  Indeed,  accident 
statistics  (e.g.,  NTSB,  1989)  suggest  that  they  often 
do  not  make  the  correct  choice  in  these  critical  situa¬ 
tions.  An  instrument  capable  of  reliably  detecting 
individual  differences  in  decision-making  skills  could 
therefore  have  a  substantial  impact  on  aviation  safety 
by,  for  example,  identifying  those  individuals  most  at 
risk  for  a  decision-related  accident  or  incident  and 
therefore  most  in  need  of  remedial  decision-making 
training.  Whether  the  current  instrument  has  that 
capability  remains  unknown,  but  this  will  be  ad¬ 
dressed  in  future  research. 
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APPENDIX  A 


EXPERT  PILOT  PANEL  DATA  FORM 


3«  In  the  cvaiing  after  an  exhausting  three  day  business  meeting  at  a  downtown  hotel,  you  have  loaded  your  rental  airplane  at  the 
Downtown  Airpoit  and  prepare  to  file  your  flight  plan  for  the  two  hour  flight  home  when  you  discover  you  left  your  only  pair  of 

reading  gliwry*  in  the  meeting  room  at  the  hotel  You  have  no  problem  seeing  the  pand  gages,  or  distance  vision,  but  can't  read  a 
irmp  or  chart  Weather  is  solid  VFR  and  if  you  depart  within  the  next  20  minutes  you  wili  arrive  at  your  home  airport  before  daiit  You 
decide  to: 


Rank  Order 
12  3  4 


1234: 


12  3  4 


12  3  4 


Ahemalhre 

a.  Depart  and  fly  home. 


b.  Call  the  hotel,  if  they  have  your  glasses  go  get  them  and  fly  home  late  this 
evening. 


c.  Call  the  hotel,  if  they  do  not  have  your  glasses,  buy  a  cheap  pair  from  an  all 
ni^t  drug  store  and  fly  home  tonight 


d.  Call  the  hotel,  if  they  have  your  glasses,  go  get  them,  spend  the  night  and  fly 
home  in  the  morning. 


%PUiuibIe  %IUslc 


Airport 

Runway 

24hr  Tower 

AKSA 

Lighted 

Runway 

Telephone 

Available 

Maintenance 

i  Regional  Airport 

i 

8800x150 

7753x150 

Yes 

Yes 

Yes 

Yes 

24  hrs 

4.  You  are  cruising  at  4500  feet  on  top  of  a  thin  haze  layer  with  the  outside  air  temperamre  at  65  degrees.  It  has  been  twenty-five  hours 
since  the  engine  was  overhauled  and  the  run-up  check  was  well  within  limits.  The  engine  slowly  loses  RFM  with  no  indications  of  oil  or 
fuel  problems.  You  suspect  carburetor  icing  and  pull  on  the  carb  heat  The  engine  backfires,  vibrates  and  loses  E^M  fast  You  decide  to: 


Rank  Order  Alternative 


•/•Plausible  V.Risk 


12  3  4 


12  3  4 


12  3  4 


12  3  4 


a.  Puli  out  the  mixture,  stop  the  engine  and  check  the  fuel  selector  valve,  mag 
switch  settings  and  declare  an  emergency. 


b.  Push  in  the  carb  heat,  keep  the  engine  running  and  divert  to  the  closest  airfield 


c.  Keep  the  carb  heat  on  and  see  what  happens. 


d.  Push  in  the  carb  heat,  keep  the  engine  at  idle,  declare  an  emergency  and  ask  for 
advice. 
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FEDERAL  AVIATION 
ADMINISTRATION 


GENERAL  AVIATION  PILOT  STUDY 


Tliit  daU  ooUcpiioB  m  wvaed  iiadcr  0MB  Approval  Number  2120-03(7.  The  public  ii^avting  bur  dm  for  thii  colledion  of  infoimatiDii  ■ 
stanatsd  to  ivvage  one  hoir  par  mpoiuc  If  you  wah  to  oomment  oa  the  accuracy  of  admilc  or  make  suggeetnu  for  reducing  iKia 
burden,  plcaae  dir^  your  oonunmla  to  the  FAA  at  the  following  addrai: 

Depaitmeitt  ofTmupartctioa 
Fedaal  Aviation  Admiaiatralion 
Officeof  Aviatim  Medidne.  AAM-240 
Waihii«ton,OC20S91 


FEDERAL  AVIATION  ADMINISTRATION  PILOT  SURVEY 

You  have  been  selected  to  participate  in  a  Federal  Aviation  Administration  ^AA)  study  deagned  to 
identify  areas  of  judgement  where  training  would  reduce  the  risk  of  incidents  or  aeddents.  You  are  especially 
valuable  in  this  research  as  your  expressed  willingness  to  participate  will  allow  us  to  examine  pilot  judgement 
across  re^ons  of  the  country,  different  age  and  levels  of  experience.  We  greatly  appreciate  your  time  and  effort 
in  completing  this  survey.  We  think  you  will  find  it  interesting. 

This  survey  consists  of  two  parts.  The  first  part  is  an  experience  questionnaire  to  enable  us  to  quantify 
the  experience  fitetors  of  the  pilots  partidpating  in  this  study.  Notice  that  your  name  or  other  personal 
identifying  information  is  not  asked  for  nor  recorded.  There  will  be  no  way  to  match  your  survey  response  with 
you. 

Please,  complete  the  Pilot  Information  Form  below.  Enter  or  circle  the  response  accurate  for  your 
aviation  experience.  Be  sure  and  drcle  all  ratings  you  have  had,  not  just  the  highest. 


PILOT  INFORMATIOJ 
CSkIs  or  mer^  correct  resi 

lonse 

Pilot  Certificates  Hdd; 

PP  COM  ATP  CFI  CFH 

B 

Instrument  Rating? 

YES  NO 

Engiiin  Ratiiig 

SINGLE  MULTI 

Ago: 

1  SUlA  of  Raaidence: 

Sex: 

M  F 

Ever  a  Military  Pilot? 

YES  NO 

Medical: 

Class  I  Class  II  Class  in 

FLYING  HISTORY 

Enter  the  aonrooriate  nunibers  to  the  best  ofyoiir  abiRfy 

Years  as  a  Pilot: 

Total  Houn: 

Weather  Hours: 

Hours  of  Ihstnictioa: 

Ni^  Houn: 

;  FLYING  HOIJRSINYHEIJ^ 

.  .  . 

Total  Houn: 

Day  Houn: 

Weather  Houn: 

Houn  of  Inattucdoci: 

Night  Houn; 

5  AmCRAFTI^ 
Enter  the  dau  for  ihe  ai 

RMAHON 

reratt  you  most  frequently  fiv 

Number  of  Ezigmea: 

DFR  Capable? 

YES  NO 

Aulo-Pilot? 

YES  NO 

Weather  Radar? 

YES  NO 

Trafispooder? 

YES  NO 

Do  You  Own  ihU  Aircraft? 

YES  NO 

Hours  of  Cruise  Capable: 
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The  second  part  is  a  set  of  scenarios  developed  from  real  experiences  reported  to  the  FAA  through  previous 
studies.  These  scenarios  contain  a  stem  ^ch  sets  the  scene,  and  four  courses  of  action  which  can  be  taken  in 
response  to  the  problem  posed  in  the  stem.  Additional  information  including  a  diagram  of  each  airfield  (except 
for  military  installations)  is  provided,  .^tfield  data  relevant  to  each  specific  scenario  is  provided  in  a  table 
immediately  following  the  possible  actions.  For  each  of  the  51  scenarios  shown  in  Part  2  of  the  survey  on  pages 
1  to  14,  do  the  following  steps: 

First,  carefully  read  the  scenario  and  the  four  listed  Alternative  responses.  Assume  you  have  leased  the 
Cessna  172  shown  on  the  flyer  from  Aircraft  Rental  and  Leasing.  Feel  free  to  use  the  airfield  diagrams  for 
assistance  in  understanding  the  problem. 

Second,  based  on  your  experience,  decide  which  of  the  alternatives  you  would  most  likely  selea  as  your  first 
course  of  action,  whk  would  be  your  second  choice,  then  your  third  and  fourth  if  you  were  the  pilot  in  the 
scenario.  To  the  left  of  each  choice  are  the  numbers  (1  2  3  4),  circle  the  1  next  to  the  choice  you  selected  as 
your  first.  Next  select  the  choice  you  consider  second  and  circle  the  2  to  the  left  of  the  choice,  then  circle  your 
third  and  fourth  choices. 

Third,  check  that  you  have  circled  only  one  1, 2,  3,  or  4  for  each  alternative. 

Last,  put  the  completed  survey  in  the  pre-addressed  envelope  and  drop  it  in  the  mail.  NO  POSTAGE 
NECESSARY! 


This  is  not  a  test  There  are  no  right  or  wrong  answers,  and  you  are  not  asked  to  do  anything  illegal.  This  is  not 
a  test  of  rules  but  a  survey  of  pilots  whd  have  *been  there,  done  that*  and  have  survived  to  pass  lessons  learned 
from  that  experience  to  others  just  starting  their  flying  careers.  These  scenarios  were  developed  from  real 
incidents  reported  in  previous  surveys  where  the  pilot  learned  a  valuable  lesson  and  was  willing  to  share  with  the 
aviation  community. 

Here  is  an  example  of  what  a  properly  completed  question  in  the  survey  should  look  like. 


IS.  Your  airplane  battery  is  dead  and  you  are  alone  at  the  airfield.  To  start  your  airplane  you  decide  to; 


Rank  Order  Aftcniatire 


I  2(3)4 


1  2  30 


&.  Get  your  jumper  cables  and  move  the  plane  close  enough  to  the  172  parked 
next  to  you  and  jump  scait  the  engine. 

b.  Prime  the  engine,  turn  on  the  master,  set  the  mag  switch  to  both,  slightly  crack 
open  the  throttle  and  swing  the  prop. 

c.  Connect  the  jumper  cables  from  the  airplane  to  your  car  and  see  if  it  will  stan. 

d.  Take  the  battery  out  and  take  it  to  be  charged  by  an  A&?  mechanic. 


Review  the  reference  material  on  the  next  six  (6)  pages. 
Following  your  review,  turn  the  last  reference  page  and 
begin  Part  2  by  rating  Items  1-51  on  pages  1-14. 
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1.  You  arc  flying  an  “Angel  FUghf*  with  a  nurse  and  non-cndcal  child  patient  to  meet  an  ambulance  at  a  downtown  regional 
airport.  You  filed  VFR,  it  is  11:00  P.M.  on  a  clear  night  when  at  60  NM  out  you  notice  the  ammeter  reading  zero  and  correctly 

deduce  the  alternator  has  felled  Your  best  guess  is  that  you  have  from  15  to  30  minutes  of  battery  power  remaining  Youdecideto: 


Sank  Order  Attemative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Declare  an  emergency,  turn  off  all  electrical  systems  except  for  1 NAVCOM  and  transponder  axul  continue 
to  the  Regional  Airport  as  planned. 

b.  Declare  an  emergency  and  divert  to  the  Planter’s  County  Airport  which  is  clearly  visible  at2o’clodc,7 
NM. 

c.  Declare  an  emergency,  turn  off  all  electrical  systems  except  for  1  NAVCOM,  instrument  panel  lights, 
intercom  and  transponder  and  divert  to  the  Sout^ide  Business  Airport  which  is  40  NM  straight  ahead 

d.  Declare  an  emergency,  tom  off  all  electrical  systems  except  for  1  NAVCOM,  instnimeot  panel  lights, 
intercom  and  transponder  and  divert  to  Draper  Air  Force  Ba»  which  is  10  o’clock  at  32  NM. 


Airport 

Runway 

24hr  Tower 

ARSA 

ISSS3I 

Telephone 

Available 

Maintenance 

Regional 

8800x130 

Yes 

Yes 

Yes 

Yes 

24  hrs 

7753x150 

Planters  County 

3200x75 

No 

No 

Yes 

Yes 

0700-1800 

Southside  Business 

4835x100 

Yes 

Yes 

Yes 

Yes 

0700-1800 

4129x100 

Draper  AFB 

11500x300 

Yes 

No 

Yes 

Yes 

None 

2.  You  are  solo  on  a  late  night  cross  country  cruising  VFR  at  9000  feet  with  two  hours  left  to  your  destination  when  you  become 
vciy drowsy.  Youdecideto: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Direct  the  cold  air  vent  onto  your  face,  sing,  keep  moving  about,  anything  you  can  to  keep  awake. 

b.  Land  at  an  airfield  8  miles  ahead,  get  a  motel  room  and  call  it  a  night. 

c.  Descend  and  continue  flying  at  a  lower  altitude. 

d.  Land  at  the  airstrip  ahe^  walk  around,  then  takeoff  and  continue. 


3.  In  the  evening  after  an  exhausting  three  day  business  meeting  at  a  downtown  hotel,  you  have  loaded  your  rental  airplane  at  the 
Downtown  Airport  and  prepare  to  file  your  VFR  flight  plan  for  the  two  hour  flight  home  when  you  discover  you  left  your  only  pair  of 
reading  glasses  in  the  meeting  room  back  at  the  hotel.  You  have  no  problem  seeing  the  panel  gages,  or  distance  vision,  but  can’t 

rcada  maporchart.  Weather  is  solid  VFR  and  if  you  depart  within  the  next  20  minutes  you  will  arrive  at  the  Regional  Airport 

before  dark.  You  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 

12  3  4 


a.  Depart  and  fly  home. 

b.  Call  the  hotel,  if  they  have  your  glasses  go  get  them  and  fly  home  late  this  evening. 

c.  Call  the  hotel,  if  they  do  not  have  your  glasses,  spend  the  night,  have  a  pair  expressed  to  you  and  fly  home 
tomorrow. 

d.  Call  the  hotel,  if  they  have  your  glasses,  go  get  them,  spend  the  night  and  fly  home  in  the  morning. 


Airport 

Ruuway 

24hr  Tower 

ARSA 

Lighted 

Runway 

Telephone 

Available 

Maintenance 

Regional  Airport 

8800x150 

Yes 

Yes 

Yes 

Yes 

24  hn 

7753x150  1 
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4.  You  are  cniisin^  at  4S00  feet  ou  top  of  a  thin  haze  layer  with  the  outside  air  teu^ratuie  at  65  degrees.  It  has  been  twenty-five 
hours  since  the  engine  was  overhauled  and  the  run-up  check  was  well  within  limits.  The  engine  slowly  loses  RPM  with  no 
indications  of  oil  or  fuel  problems.  You  suspect  carburetor  icing  and  pull  on  the  carb  beat  The  engine  backfires,  vibrates  and  loses 
RPM  fast.  You  decide  to: 


Rank  Order  Alternative 


12  3  4 

12  3  4 
12  3  4 
12  3  4 


a.  Puli  out  the  mixture,  stop  the  engine  and  dieck  the  fuel  selector  valve,  mag  switch  settings  and  declare  an 
emergency. 

h.  Push  in  the  carb  heat,  ke^  the  engine  running  and  divert  to  the  closest  airfield. 

c.  Keep  the  caib  heat  on  and  see  what  happens. 

d.  Push  in  the  carb  heat,  keep  the  engine  at  idle,  declare  an  emergency  and  ask  for  advice. 


5.  You  are  preparing  to  enter  the  VFH  traffic  pattern  at  the  Regional  Airport  and  hear  the  tower  report  winds  from  2S0  at  15  knots, 
andtheyare  vectoring  traffic  to  the  primary  SSOO  ft  runways  35.  A  Piper  Gierokee  asks  to  use  the  7753  x  150  runway  27,  The 
Cherokee  is  told  the  runway  is  not  active,  but  to  you  it  looks  OK.  Youdecideto: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Accqjt  clearance  to  runway  35  and  follow  the  traffic. 

b.  Ask  to  use  runway  27. 

c.  Insist  on  using  runway  27  stating  that  the  crosswinds  are  unsafe  for  you  to  use  runway  35. 

d.  Divert  to  the  Southside  Business  Airport  where  the  runway  is  almost  directly  aligned  with  the  wind. 


Airport 

Runway 

24hr  Tower 

ARSA 

Lighted 

Runway 

Telephone 

Available 

Maintenance 

Regional  Airport 

8800x130 

Yes 

Yes 

Yes 

Yes 

24  hrs 

7753x150 

Southside  Business 

4835x100 

Yes 

Yes 

Yes 

Yes 

0700-1800 

4129x100 

6.  It  is  a  cool  clear  summer  afiemoon  with  no  wind  when  you  arrive  in  ARS  A  going  to  the  Regional  Airport.  You  realize  you  arc 
going  to  be  spaced  4  miles  behind  a  commercial  727  on  final  to  runway  17.  You  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Stay  high  on  the  glide  slope  and  land  past  where  you  saw  the  727  touchdown. 

b.  Ask  for  a  360  turn  to  increase  the  spacing. 

c.  Ask  to  land  on  runway  09. 

d.  Ask  for  a  low  approach  and  a  visual  pattern  to  runway  17. 


Airport 

Runway 

24hr  Tower 

ARSA 

lighted 

Runwav 

Telephone 

Available 

Maintenance 

Regional  Airport 

8800x150 

Yes 

Yes 

Yes 

Yes 

24  hrs 

7753x150 

7.  To  prepare  for  when  marginal  VFR  weather  makes  it  difficult  to  return  to  your  home  airfield  (uncontrolled),  you  practice  in  VFR 
conditions: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  An  unofficial  locally  devised  arrival  to  the  pattern. 

b.  Have  devised  your  own  arrival  route  to  the  pattern  or  runway. 

c.  Practice  a  published  IFR  approach. 

d.  Don’t  do  anything. 
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You  as  the  pilot-ia-command  (PIC)  arc  going  to  fly  your  old  instructor  pilot  to  the  Planters  County  Airport  so  he  can  pickHq>  an 
airplane  coining  out  of  maintena"^  and  give  it  a  functional  check  flight.  Both  of  you  arrive  at  the  airport  later  than  you  planned 
and  he  offers  to  do  part  of  the  preflight  You  decide  to: 

Rank  Order  Alternative 

1  2  3  4  a.  Do  the  planning,  filing  and  preflight  together, 

1  2  3  4  b.  Have  hiin  get  the  weather  NOTAMS  and  file  the  flight  plan  tvhile  you  preflight  the  airplane. 

1  2  3  4  c.  Have  him  preflight  the  airplane  while  you  get  the  weather,  NOTAMS,  and  file. 

1  2  3  4  d.  Ask  him  who  should  do  what 


9.  You  are  at  a  small  airport  with  minimal  facilities  and  at  the  end  of  your  walk  around  predict  the  flaps  refuse  to  retract  from  30 
degrees.  It  was  a  planned  three  hour  flight  back  home  to  the  Regional  Airport  The  attendant  says  he  has  seen  this  problem  before 
and  it  is  the  limit  switch  sticking.  There  is  no  A&P  here  but  there  is  an  A&P  at  an  airport  35  miles  up  the  road.  The  attendant  says 
he  knows  where  a  switch  for  this  exact  model  172  can  be  quickly  picked-up  and  he  could  install  it  He  says  he  also  could  reach  up 
through  the  inspection  port  and  free  the  switch  enough  to  raise  the  flaps,  but  canr.ot  guarantee  they  will  work  when  airborne.  You 
call  the  rental  agency  and  get  their  answering  machine  -  you  arc  on  your  own.  You  decide  to: 

Rank  O  rder  Alternative 

1  2  3  4  a.  Leave  the  flaps  down  and  fly  to  the  nearby  (35  miles)  airport  and  have  an  A&P  fix  the  problem. 

1  2  3  4  b.  Have  the  attendant  reset  the  switch,  get  the  flaps  up  and  fly  back  to  Regional. 

1  2  3  4  c.  Have  the  attendant  change  the  switch,  chock  it  out  then  fly  home  and  have  the  rental  agency  inspect  the 

work. 

1  2  3  4  d.  Wait  until  the  rental  agency  can  fly  an  A&P  in  and  change  the  switch. 

10,  You  arc  planning  a  night  cross  country  down  to  Florida  and  winds  aiul  weather  favor  cruising  around  8000  feet  MSL.  The 
forecast  winds  and  visibility  will  enable  you  to  make  your  destination  (solid  VFR  weather)  with  a  60  minute  fiicl  reserve  in  one  hop. 
You  decide  to: 

Rank  Order  Alternative 

1  2  3  4  a.  Let  down  early  and  cruise  in  at  a  much  lower  altitude  if  fuel  permits. 

1  2  3  4  b.  Stay  at  altitude  as  long  as  possible  before  performing  an  en  route  descent. 

1  2  3  4  c.  sure  there  is  a  working  oxygen  system  on  board  in  case  you  need  it 

1  2  3  4  d.  Plan  to  use  oxygen  for  this  flight 
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7Tr«  next  four  scenarios  (11  •  14)  are  related  and  occur  on  the  same  flight: 


IL  You  ait  at  the  College  Aiipoft  to  pick-up  three  passengers  and  their  baggage  and  letum  them  to  the  RegLonal  Airpoit  Before 
refueling  you  add  up  the  weights  and  find  with  full  foci  (40  CiaUoos)  your  load  will  be  40  pounds  over  the  book's  max  gross  rveight 
Weather  for  the  3 :00  FM  return  trip  is  forecast  at  6000*  scattered,  visibility  ICH  and  the  winds  at  5000  feet  cruising  altitude  will  net 
a  10  knot  tailwind.  Using  the  Operator's  Manual  fuel  consumption  rate  aiul  the  tailwind  you  correctly  calculate  it  will  take  34 
gallons  of  1(X)LL  to  land  at  Regional  with  exactly  30  minutes  reserve.  You  will  overfly  the  Justin  County  Airport  and  could  land  for 
fuel  as  a  backup,  You  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Load  34  gallons  and  file  a  flight  plan  direct  to  the  Regional  Airport 

b.  Load  40  gallons  and  file  direct  to  the  Regional  Airport 

c.  Loadonly  34  gallons  and  file  to  the  Regional  Airport  with  a  stopover  at  Justin  County  Airport 

d.  Load  34  gallons,  do  not  file  and  see  if  the  fud  consumption  and  tailwind  hold  and  d^de  later  what  to  do. 


Airport 

Runway 

24hr  Tower 

ARSA 

Lighted 

Runway 

Telephone 

Available 

Maintenance 

Regional  Airport 

8800x150 

Yes 

Yes 

Yes 

Yes 

24hrs 

7753x150 

Justin  County  Airport 

3200x50 

No 

No 

Yes 

Yes 

0700-1800 

College  Airport 

5000x100 

Yes 

No 

Yes 

Yes 

24hrs 

4099x100 

12.  You  have  taken-off  firom  the  College  Airport  and  an  en  route  weather  check  has  a  late  afternoon  thunderstorm  approaching  the 
Regional  Airport  from  the  opposite  side  of  town.  It  is  slow  moving  and  is  expected  to  cross  the  Regional  Airport  shortly  after  your 
ETA.  You  check  and  the  fuel  consumption  and  tailwind  are  holding.  You  have  arrival  fuel  with  a  30  minute  reserve.  You  d^ide 
to: 

Rank  Order  Alternative 


12  3  4 
12  3  4 

12  3  4 
12  3  4 


a.  Continue  to  the  Regional  Airport  and  speed  up  a  bit. 

b.  Land  at  the  Justin  County  Airport,  add  fuel  and  continue  to  the  Regional  Airport  circling  northeast  around 
the  thunderstorm. 

c.  Land  at  the  Justin  County  Airport  and  wait  until  the  weather  passes. 

d.  Land  at  the  Justin  County  Airport,  add  fuel  and  continue  to  the  Regional  Airport  circling  southwest  around 
the  thunderstorm. 


13,  Your  friends  persuaded  you  to  land  at  the  Justin  County  Airport  You  plan  to  fill  each  tank  half  full  to  keep  the  weight  in  the 
utility  category.  The  thunderstorm  remains  slow  moving,  is  over  the  Regional  Airport  on  a  path  to  the  Justin  County  Airport  and  is 
growing  in  size  and  intensity.  It  is  6;00  PM,  getting  dark,  the  storm  can  be  seen  approaching  and  the  attendant  is  leaving  but  will 
give  everyone  a  lift  into  Driskill  City.  You  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Takeoff  for  the  Regional  Airport  circling  around  the  thunderstorm  and  coining  in  behind  it 

b.  Wait  with  the  airplane  until  the  weather  passes,  then  fly  into  the  Regional  Airport. 

c.  Leave  the  passengers  and  baggage  and  fly  the  airplane  anywEiere  away  from  the  path  of  the  storm. 

d.  Leave  the  airplane  and  either  get  a  room  in  Driskill  City  or  call  and  have  someone  drive  out  from  the  Big 
City  and  pick-up  all  of  you. 


Airport 

Runway 

24hr  Tower 

ARSA 

Lighted 

Runway 

Telephone 

Available 

Maintenance 

Regional  Airport 

8800x150 

Yes 

Yes 

Yes 

Yes 

24hrs 

7753x150 

Justin  County  Airport 

3200x50 

No 

No 

Yes 

Yes 

0700.1800 
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14.  Your  passenger  decided  to  go  into  Driskill  City  and  have  someone  drive  out  and  pick  them  up. 

The  approaching  thimdcretormconUnucs  to  grow  with  reports  of  80  MPH  wind  gusts  and  large  hail.  To  save  the  airplane  you  take 

off  and  circle  northeast  around  the  storm  to  avoid  any  possible  wall  cloud.  Y ou  plan  to  stay  at  least  20  NM  away  from  the  storm. 
You  level  at  3000  feet  with  the  storm  on  the  left  wingtip  and  the  Regional  Airport  VOR  DME  reading  27  NM.  An  undercast  begins 
to  develop  and  die  lights  of  the  city  fade.  You  decide  to: 

Rank  Order  Alternative 

1  2  3  4  a.  Hold  altitude,  start  flying  counterclockwise  around  the  27  NM  arc  to  clear  air  behind  the  storm. 

1  2  3  4  b.  Contact  Big  City  Approach  control  and  tell  them  your  problem  and  ask  for  vectors  avoiding  the  storm  to 

any  clear  airport. 

1  2  3  4  c.  Descend  slowly  to  MEA  (1000  feet)  and  establish  VFR  conditions  before  entering  ARS  A  and  contacting 

Big  City  Approach. 

1  2  3  4  d.  Make  a  right  turn,  put  the  storm  on  the  tail  as  best  you  can,  find  clear  air,  and  orbit  until  the  Regional 

Airport  reports  VFR  conditions. 


15,  Bad  weather  forced  you  to  cancel  flying  your  boss  into  another  city  where  he  is  to  address  a  convention.  There  are  openings  on 
a  flight  going  to  the  sa™  city  departing  from  the  airline  terminal  on  the  other  side  of  the  airport  in  15  minutes.  It  will  take  too  long 
to  c^l  a  taxi  so  he  asks  you  to  run  him  over  to  the  terminal  in  the  172.  You  decide  to: 


Rank  Order  Alternative 

1  2  3  4  a.  Start  the  engine  and  ask  ground  control  for  permission  to  taxi  to  the  back  of  the  terminal,  drop  off  a 

passenger  and  taxi  back  to  the  FBO  ramp. 

1  2  3  4  b.  Slart  and  ask  ground  control  for  permission  to  taxi  around  the  airport  for  a  maintenance  check  and 

conveniently  drop  the  boss  off  near  the  terminal. 

1  2  3  4  c.  Say  you’re  sorry  but  it  is  illegal  for  you  to  deliver  passengers  to  the  back  side  of  the  terminal  and  help  find  a 

ride  through  the  FBO. 

1  2  3  4  d.  Ask  ground  control  if  there  is  any  way  a  representative  firom  the  airline  could  meet  you  at  a  door  to  the 

ramp  and  escort  the  boss  into  the  terminal. 


Airport 

Runway 

24 hr  Tower 

ARSA 

JJgfated 

Runway 

Telephone 

Available 

Maintenance 

Regional  Airport 

8800x150 

7753x150 

Yes 

Yes 

Yes 

Yes 

24  hrs 

16.  You  have  entered  the  left  turn  VFR  traffic  pattern  for  runway  27  at  the  Regional  Airport  and  hear  muffled  bangs  and  feel  serious 
vibrations.  New  dents  in  the  top  of  the  engine  cowling  confirm  a  cylinder  has  failed.  The  winds  arc  070/5  and  you  arc  just 
approaching  extended  centerline  of  runway  35.  You  decide  to: 


Rank  Order  Alternative 


1  2  3  4  a  Declare  an  emergency,  shutdown  the  engine  and  turn  left  to  land  in  the  crosswind  on  nmway  35. 

1  2  3  4  b.  Declare  an  emergency,  shutdown  the  engine  and  turn  left  180  degrees  to  land  on  the  grass  on  the  left  side  of 

runway  27. 

1  2  3  4  c.  Declare  an  emergency,  shutdown  the  engine  and  turn  left  and  stretch  the  turn  to  land  on  runway  27.  ’ 

1  2  3  4  d.  Declare  an  emergency,  shutdown  the  engine  and  land  straight  ahead  on  the  taxiway  on  the  south  side  of 


BIO 


17.  While  en  route  you  want  to  find  out  what  is  going  on  along  the  weather  pattern  you  observe  ahead  You  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Call  an  airport  tower  bdow  and  ask. 

b.  Call  FSS  and  ask. 

c.  Find  the  ATC  frequency,  call  and  ask  them. 

d.  Identify  an  airplay  ahead  and  ask  for  a  PIREP. 


18.  You  have  been  away  for  five  days  and  are  returning  to  the  Justin  County  Airport  to  return  the  172  to  the  fiiend  who  loaned  it  to 
you  and  pick  your  car.  The  weather  is  clear  and  cold  as  forecast  and  a  white  blanket  covers  the  ground  When  you  near  the 
Justin  C^ty  Airpoit,  you  notice  the  runway  has  not  been  cleared.  You  cannot  tcU  how  deep  the  snow  is,  but  the  county  road  is 
&irly  clear  except  for  a  small  strip  of  snow  ^wn  the  middle.  You  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 

12  3  4 
12  3  4 


a.  Divert  to  the  Regional  Airport  and  return  the  plane  another  day. 

b.  Land,  but  hold  the  airplane  off  the  runway  until  is  in  a  full  st^,  and  keep  the  nose  wheel  off  the  ground  as 
long  as  possible. 

c.  Make  a  normal  landing,  but  don't  touch  the  brakes  unless  absolutely  necessary. 

d.  First,  do  a  touch  and  go  to  see  how  deep  the  snow  is  keeping  your  airspeed  up  and  the  nose  wheel  off  the 
ground.  If  control  is  no  problem,  land. 


Airport 

Runway 

24hr  Tower 
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No 

No 
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19.  You  just  checked  in  with  approach  on  124.9  after  a  long  solo  cross  countiy  before  entering  ARSA.  Listening  to  traffic  being 
vectored,  it  becomes  apparent  the  FedEx  flights  are  ail  returning  just  ahead  of  you,  and  it  could  be  20  minutes  before  you  land  at  the 
Regional  Airport  where  you  rented  this  airplane.  The  problem  is  you  have  to  urinate  and  can't  wait  the  20  minutes  plus  taxi  time. 
Your  trusty  relief  bottle  is  in  the  pouch  behind  the  ffont  passenger  seat.  You  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Continue  to  follow  vectors,  get  out  the  bottle  and  use  it 

b.  Tell  approach  of  your  problem  and  request  landing  priority. 

c.  Get  clearance  outside  ARSA,  find  a  saie  area  to  loiter  and  use  the  bottle. 

d.  Divert  to  the  Justin  County  Airport  which  you  overflew  16  NM  back  and  land. 


Airpon 

Runway 

24hr  Tower 
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No 

No 
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20,  You  have  announced  on  CTAF  and  upon  starting  your  turn  to  base  you  see  another  aircraft  on  a  straight-in  which  will  conflict. 
You  decide  to: 


Rank  Order 
12  3  4 
12  3  4 
12  3  4 
12  3  4 


Alternative 

a.  Continue  on,  flash  your  landing  lights. 

b.  Do  a  level  left  360  degree  turn  for  spacing. 

c.  Turn  right,  exit  the  pattern  and  re-enter. 

d.  Extend  your  downwind  to  take  spacing  behind  the  sttaight-in. 
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21.  You  are  on  short  final  at  an  uncontrolled  aiificld  with  one  other  airplane  in  the  pattern  and  have  not  complied  your  checklist 
You  decide  to: 


Rank  Order  Alternative 


1  2  3  4  a.  Check  the  flap  setting  and  land. 

1  2  3  4  b.  Go  around. 

1  2  3  4  c.  Check  the  mixture  and  land. 

1  2  3  4  d.  Keep  your  head  out  of  the  cockpit  and  land. 

22,  The  early  afternoon  ramp  temperature  at  the  Regional  Airport  is  already  94  degrees  and  the  inside  of  the  ^laiK  is  like  an 
oven.  You  are  flying  your  mother  up  to  your  sister's  to  be  with  her  during  surgery  this  evening.  Your  mother  is  afiraid  the  hot 
airplane  will  her  airsick,  so  would  you  please  spend  as  little  time  on  the  ground  in  the  heat  as  possible.  You  are  parked  on  the 
Aircraft  Rental  and  Leasing  ramp  and  sec  10  aircraft  lining  up  on  the  south  taxiway  for  a  nmway  09  takeoff.  Winds  are  060/12. 
You  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Start  and  follow  the  ttaffic  to  runway  09. 

b.  Start  and  ask  for  a  runway  35  takeoff. 

c.  Start  and  request  an  intersection  takeoff  on  nmway  09. 

d.  Delay  going  to  the  airplane  until  traffic  has  cleared. 


Airport 
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23.  You  are  at  a  rancher  fiicnd's  private  airstrip  and  he  asks  you  to  fly  him  in  bis  172  to  check  on  his  cattle  (his  foot  is  in  a  cast). 
Weather  appears  to  be  around  3000  feet  overcast  with  widely  scattered  showers  within  eyesight.  You  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Go  with  what  you  sec. 

b.  Telephone  the  FSS  for  a  briefing. 

c.  Call  to  the  local  radio  station  for  the  forecast. 

d.  Ask  the  rancher  what  you  should  do. 


24.  Your  BFR  is  coming  due  and  you  feel  a  little  nisty.  You  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Look  for  an  instructor  who  can  both  get  you  current  and  administer  the  BFR. 

b.  Hire  an  instructor  to  get  ready  but  have  a  different  instructor  give  the  BFR. 

c.  Practice  solo  and  then  take  the  BFR 

d.  Have  a  pilot  friend  work  with  you  to  get  ready  for  the  BFK 


25.  You  have  stopped  for  gas  at  a  small  airstrip  and  are  loaded  with  cargo.  You  can  only  fuel  to  30  gallons  in  the  tanks  arid  keep 
under  the  airplane's  max  gross  weight  A  30  gallon  load  will  just  enable  you  to  make  it  home  with  the  required  reserve  without 
another  fuel  stop.  You  have  no  calibrated  dip  stick  and  have  a  new  attendant  to  pump  the  gas  for  you.  You  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Fill  it  using  the  gages  to  read  3/4  full. 

b.  Fill  it  full  then  have  the  attendant  drain  off  the  difTcrcncc  between  the  tanks  capacity  and  30  gallons. 

c.  Leave  the  problem  entirely  to  the  attendant 

d.  Use  a  calibrated  stick  the  attendant  has  in  the  office  that  is  firom  an  earlier  model  172 
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26,  The  weather  i$  stuck  in  the  summertime  high  mode  with  clear  mornings,  ha;^  afternoons,  pu%  clouds  scattered  at  5500  feet 
AGL  with  visibility  at  7  miles  or  more.  When  you  go  cross  country  in  these  weather  conditions  you  usually  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Don't  file  but  fly  airways. 

b.  File  VFK  and  stay  off  airways. 

c.  Don't  file  and  don’t  use  airways. 

d.  File  VFR  on  airways  as  much  as  possible. 


27.  Take-ofif  and  en  route  weather  arc  VFR  with  a  dry  line  scheduled  through  your  destination  about  your  ETA.  It  may  push  some 
thunderstorms  ahead  of  it  so  your  weather  briefing  ends  with  “VFR  flight  is  not  iccommcndetL"  There  arc  several  good  alternate 
airfields  along  the  route  of  flight  and  beyond  your  destination.  You  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Go  without  filing  a  flight  plan. 

b.  File  VFR  to  an  airport  short  of  your  destination,  land  and  let  any  weather  pass  over. 

c.  Delay  your  departure  until  the  “VFR  flight  is  not  recommended'’  statement  is  removed  from  the  forecast. 

d.  File  VFR  to  your  dcstinatioiL 


Airport 
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Runway 

Telephone 

Available 

Maintenance 
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28.  You  have  been  cruising  at  8500  feet  for  four  hours  and  arc  preparing  to  do  a  feirly  steep  en  route  descent  directly  onto  shon 
final  at  the  Regional  Airpon.  Weather  en  route  was  a  -30  degrees  but  the  warm  sun  made  the  flight  most  comfortable.  Regional 
weather  is  54  degrees,  dew  point  of  52  degrees  and  calm  winds.  You  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Expedite  the  descent  to  give  as  much  time  on  final  approach  as  possible. 

b.  Make  sure  the  carb  heat  and  window  defrosters  are  on  and  set  at  high. 

c.  Slow  the  descent  rate  to  allow  the  airframe  to  warm  before  landing. 

d.  Find  a  rag  and  keep  it  handy  in  case  you  have  to  clean  windows. 


29,  The  weather  for  departure  and  the  first  half  of  your  four  hour  cross  country  was  slightly  better  than  marginal  VFR.  You  made  it 
off  and  have  leveled  at  cruise  altitude  in  VFR  conditions  and  arc  preparing  to  check  in  with  Flight  Watch  You  suspea  they  will  ask 
youforaPIREP  to  check  their  forecast.  You  decide  to: 


Rank  Order  Alternative 


12  3  4 

12  3  4 
12  3  4 

12  3  4 


a.  Calculate  your  drift,  determine  the  winds,  make  note  of  the  cloud  cover  and  types,  and  note  the  OAT  to  be 
ready  when  they  ask. 

b.  Beg  off  -  telling  them  you  have  your  hands  full  and  can't  take  the  time. 

c.  Expect  to  give  at  least  your  position,  cloud  bases  and  tops,  visibility  and  relate  any  deviations  between  what 
you  saw  and  what  was  forecast. 

d.  Prepare  to  cither  confirm  the  accuracy  of  their  forecast,  or  tell  of  the  observable  differences. 
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30.  You  have  off  fiom  a  resort  beaded  back  home  when  after  30  minutca  you  notice  you  cannot  identify  any  landmaika  or 

checkpoints  on  your  route  of  flight  You  check  the  instniments  and  find  the  cage  knob  on  the  W4I  not  fuUy  in  and  toirned.  When 

you  uncage  it  it  swings  wUdly  and  points  back  in  the  opposite  direction  you  were  si^posed  to  track.  You  decide  to: 


Rank  Order  Alternative 


12  3  4 

12  3  4 
12  3  4 

12  3  4 


a.  Plot  out  a  reciprocal  heading  on  your  sectional,  turn  around,  call  Flight  Following  and  tcU  them  what  you 
did,  then  detennine  your  hours  of  fiiel  remaining  and  modify  your  flight  plan. 

b.  CallFlightWatc^  tell  what  you  did  and  ask  for  a  vector  l»ck  on  course. 

c.  Turn  around,  ranc<^i  your  flight  plan,  don’t  say  why,  return  to  the  airport  where  you  took^xff,  land,  fuel  and 

start  over.  .  , 

d.  Cali  Flight  Watch,  cancel  your  flight  plan,  then  plot  a  new  route  home  and  go  without  a  flight  plan. 


31.  You  have  planned  a  four  plus  hour  cross  country  and  the  weather  could  easily  force  you  into  rather  undesirable  routes  which 
would  take  you  over  rough  and  desolate  country.  To  match  the  best  weather  and  route  combination,  you  decide  to: 


Rank  Order  Alternative 


12  3  4 

12  3  4 
12  3  4 
12  3  4 


a.  Select  the  route  with  which  you  feel  the  most  comfortable  and  have  the  weather  forecaster  give  you  the 
forecast  and  if  VFR  is  not  recommended,  repeat  this  process  until  you  have  a  VFR  route. 

b.  Tell  the  forecaster  your  departure  point,  destination  and  have  him  select  the  best  route. 

c.  Give  the  forecaster  three  routes  and  have  him  give  you  the  weather  for  each  then  you  decide. 

d.  Delay  the  flight  until  you  get  VFR  weather  over  the  primary  route. 


32.  You  have  called  an  aircraft  mechanic  you  usually  use  whose  hangar  is  on  the  Justin  County  Airport,  and  arrange  to  have  your 
inoperative  landing  light  rewired.  Since  you  will  arrive  after  dark,  he  advises  you  that  the  runway  lights  are  out  and  the  black 
asphalt  runway  may  cause  you  a  problem.  You  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Ask  him  if  he  will  park  his  pickup  near  the  threshold  and  shine  his  lights  down  the  runway. 

b.  Postpone  the  repair  until  you  can  land  in  daylight. 

c.  Go  ahead  and  make  a  black-out  landing. 

d.  Postpone  the  flight  at  least  until  the  runway  lights  arc  working. 
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33.  You  aie  one  hour  into  a  three  hour  cross  country  returning  to  foe  Regional  in  foe  mid  afternoon  and  have  been  flying  at  5000 
feet  MSL  over  a  scattered  deck  of  clouds  and  find  yourself  lost.  The  cloud  deck  thins  out  to  where  you  can  sec  foe  ground  in  all 
directions.  You  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 

12  3  4 

12  3  4 


a.  Call  FSS  on  foe  radio,  report  yourself  lost,  and  have  them  give  you  a  position  fix  and  vector  back  to  course. 

b.  Find  a  prominent  landmark,  circle  it  until  you  can  find  it  on  your  sectional,  fix  your  position  and  lay  out  a 
heading  to  get  back  on  course. 

c.  Go  to  the  nearest  town,  descend  and  read  the  name  of  foe  town  on  a  water  tower  or  other  prominent 
structure,  fix  your  position  and  plot  a  heading  to  get  back  on  course. 

d.  Find  an  airfield,  land  and  ask  where  you  are,  refile,  then  continue. 
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34.  YouareflyingyourbossfromtheRcgioiial  Airport  to  Planter's  County  to  appear  as  a  witness  before  the  Grand  July.  Asyou 
stop  into  tbc  wind  for  your  engiiie  run  up,  your  boss  opens  his  door,  leans  out  and  vomits.  When  finished,  he  closes  the  door  and 
S£Qrs  “Let’s  go.”  You  have  no  ainiclc  bags  on  board  You  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Takeoff  and  fly  as  planed  to  Planter’s  County  and  find  some  suitable  airsick  bags  for  the  return  trip. 

b.  Taxi  back  to  the  FBO,  pick  up  some  airsick  bags  and  go. 

c.  Tell  the  boss  you  will  not  fly  until  he  assures  you  he  is  wdl  enough  to  make  the  flight 
d  Cancel  the  flight  and  taxi  l^k. 
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35.  You  have  paid  for  and  been  planning  this  flight  to  the  Lodge  Resort  at  the  Lake  for  six  months.  The  weather  is  forecast  good 
VFR  with  a  summer  haze  under  3000  feet  and  broken  scattered  clouds  along  the  route  of  flight  The  only  problem  is  you  know  you 
have  a  minor  summer  cold  You  can  clear  your  ears  and  only  feel  a  little  achy  with  no  headache.  You  decide  to: 

Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a  Take  the  minimal  dosage  ofcold  tablets  and  go. 

b.  Cancel  the  flight 

c.  Call  your  doctor  and  ask  for  a  prescription  for  medication. 

d  Stick  a  menthol  inhaler  in  your  pocket,  take  no  other  medication  and  go. 


3d  You  are  20  NM  outbound  from  Regional  Airport  flying  solo  to  deliver  two  coolers  of  medical  serum  to  an  American  Red  Cross 
fleld  team  when  departure  control  calls  advising  that  someone  reported  a  right  wheel  pant  was  found  off  the  departure  end  of  the 
runway  and  it  looks  like  it  may  have  separated  from  a  172  and  is  painted  a  white  similar  to  the  color  of  your  airplane.  You  neither 
heard  nor  felt  anything  unusual  on  takeoff  and  both  brake  pedals  feel  normal  when  you  apply  them.  You  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Thank  them  for  the  call  and  ask  it  be  delivered  to  Aircraft  Rental  and  Leasing  and  continue  your  departure. 

b.  Un^en  your  seatbelt,  slide  over  to  the  right  seat  and  confirm  if  it  is  yours. 

c.  Request  clearance  to  return  and  request  a  fly-by  the  tower  and  have  them  determine  if  it  is  yours. 

d.  Request  clearance  to  return  and  land  to  inspect  the  airplane. 


Airport 

Runway 

24hr  Tower 

ARSA 

Lighted 

Runway 

Telephone 

Available 

Maintenance 

Regional  Airport 

Yes 

Yes 

Yes 

Yes 

24  hrs 

7753x150 
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37.  You  Deed  to  dei»rt  the  Planter’s  County  Airport  at  6:00  AM  for  a  one  hour  flight  to  return  the  rented  aircraft  to  the  Regional 
Airport  before  7:00  AM.  You  slide  the  left  scat  back  to  climb  in  and  start  the  preflight  when  the  scat  comes  off  of  the  slide  tracks. 
You  get  the  seat  back  on  the  track  and  it  seems  to  hold.  You  notice  that  two  screws  that  hold  a  keeper  on  the  back  of  the  track  arc 
missing  and  find  one  under  the  back  seat  The  local  mechanic  will  not  arrive  for  two  or  three  hours.  Youdeddeto: 

Rank  Order  Alternative 

1  2  3  4  a.  Bonow  a  screwdriver,  put  in  the  screw  and  fly  as  is  having  the  rental  firm  checked  or  fix  the  seat  back  at 

the  Regional  Airport 

1  2  3  4  b.  Waituntilthemechanicanivcsandhavehimfixtheseat,  then  fly  home. 

1  2  3  4  c.  Skip  the  repairs  and  fly  the  trip  home  fiom  the  right  seat. 

1  2  3  4 _ d.  Go  find  a  phone,  call  Aircr^  Rental  and  Losing  and  regu^  guidance  what  to  do.  _ 

Airport  Runway  I  X4hr  Tower  TaRSA  Lighted  Telephone  Maintenance 

Runwav  Available 


38.  You  have  taken  off  solo  from  the  Regional  Airport  for  a  45  minute  flight  to  the  Planters  County  Airport  and  have  leveled  at 
3500  feet  when  you  hea*-  a  banging  start  on  the  right  side  of  the  airplane.  Everything  checks  OK  so  you  call  the  FBO  and  ask  for 
advice.  After  a  short  period  they  ask  you  to  find  both  ends  of  the  right  scat  belt.  You  can  only  find  one.  You  decide  to: 


Rank  Order  Alternative 

1  2  3  4  a.  Reach  over,  open  the  right  door,  pull  in  the  seatbelt  and  close  the  door. 

1  2  3  4  b.  Return  to  the  Regional  Airport,  land  and  pull  in  the  scat  belt 

1  2  3  4  c.  Continue  and  find  an  airspeed  where  the  banging  stops  and  continue  to  dcstinatioiL 

1  2  3  4  d.  Find  the  closest  airport  out  of  ARSA,  land  and  pull  in  the  seatbelt. 

39.  You  arc  looking  for  I72s  to  rent  You  have  decided  the  most  important  thing  to  look  for  in  a  rental  plane  is: 

Rank  Order  Alternative 

1  2  3  4  a.  The  overall  appearance,  is  it  neat  and  does  it  look  cared  for. 

1  2  3  4  b.  A  clean  engine  with  clean  oil. 

1  2  3  4  c.  New  COM/NAV  radios. 

1  2  3  4  d.  Smooth  skin,  no  dente  Of  dings. 

40.  You  have  drawn  the  cross  country  route  on  your  map  and  are  going  to  pick  your  cruising  altitude.  You  will  be  at  altitude  for 
approximately  3  hours.  You  decide  to  pick  your  cruising  altitude  primarily  based  on: 

Rank  Order  Alternative 

1  2  3  4  a.  MEA 

1  2  3  4  b.  Aircraft  cruise  performance  (tiue/fuel  bum) 

1  2  3  4  c.  Winds  aloft 

1  2  3  4  d.  WcathcrAisibility. 

41.  You  arc  going  to  ferry  an  airplane  to  the  factory  and  return  another  which  has  had  new  radios  instaUed.  You  have  completed  all 
the  flight  planning  and  decide  to: 

Rank  Order  Alternative 

1  2  3  4  a.  Have  another  pilot  (not  an  instructor)  you  trust  check  your  planning. 

1  2  3  4  b.  Find  an  instructor  to  review  your  planning. 

1  2  3  4  c.  Feel  confident  enough  not  to  need  a  review. 

1  2  3  4  d.  Ask  the  owner  of  the  airplane  to  review  your  planning. 
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42.  You  are  going  to  spend  two  houcs  ill  the  traffic  pattern.  You  plan  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Stay  in  the  pattern  at  your  home  airport. 

b.  Go  wofk  wtert  the  tr^c  is  least. 

c.  Go  to  a  strange  field. 

(L  Go  to  a  chaUenging  field  (short,  grass,  etc.). 


43.  You  arrive  at  the  rental  airplane  where  the  attendant  (whom  you  know  is  a  commercial  pilot)  says  it  is  ga.™d  to  the  top, 
preflighted  and  ready  to  start  He  will  stay  and  help  until  you  leave  the  parking  ramp.  You  decide  to; 


Rank  Order  Alternative 


12  3  4 
12  3  4 

12  3  4 
12  3  4 


a.  Thank  him,  chedt  the  fiiel  tanks,  oil ,  then  climb  in  and  start 

b.  Scan  the  airplane  for  any  obvious  errors,  climb  in  the  plane,  and  chc(^  the  fuel  gages.  If  they  show  fiiU, 
begin  your  preflight  at  engine  start. 

c.  Ta^  out  your  checklist  and  do  a  complete  preflight 

d.  Do  a  fast  walk  around  especially  checking  the  iiiel  tanks  and  caps,  oil  stick  and  all  doors  closed. 


44.  You  are  cruising  at  2500  feet  on  a  beautiful  clear  day  10  miles  out  enroute  to  the  Planters  County  Airport  with  your  best  firiend 
then  he/sbe  asks  **What  do  you  do  if  the  engine  quits?**  You  decide  to: 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Pull  the  mixture  and  show  how  the  engine  can  be  restaned. 

b.  Pull  on  the  carb  heat  bring  the  throttle  to  idle  and  demonstrate  a  forced  landing  to  a  low  approach. 

c.  Tell  your  fiiend  about  what  you  would  do. 

d.  Wait  until  you  are  over  the  uncontrolled  airfield  and  demo  a  fbiced  landing  to  a  fiill  stop. 


Airport 

Runway 

24hr  Tower 

ARSA 

Lighted 

Runway 

Telephone 

Available 

Maintenance 

Planters  County 

3200x75 

No 

No 

Yes 

Yes 

0700-1800 

45.  You  are  planning  a  cross  country  which  will  require  a  fuel  stop.  In  what  order  would  you  consider  the  following  factors  in 
selecting  the  airport  at  which  to  stop? 


Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  The  amenities  (Tcstrooms,  food  service,  loaner  car,  etc.) 

b.  Thepilotsupport&cilities(FSSaccess,  weatherstation,  etc.) 

c.  The  sire  of  the  airport  and  its  congestion  ( those  factors  thaf  make  for  slow  fuel  stops). 

d.  The  cost  of  the  fuel  or  the  method  for  payment. 
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46  Thice  of  your  fHand*  have  bought  you  a  choice  ticket  and  aie  paying  for  you  to  lent  thia  airplane  and  fly  the  four  of  you 
180  mUes  up  to  the  university  in  the  morning  lor  the  “BIG"  early  afternoon  fboAaU  game,  then  back  in  the  early  evenings 
Another  friend  wUl  meet  you  at  the  coUege  airport  and  drive  all  of  you  to  the  game  and  back.  Departure  weather  was  overcast  3000 

ft  ceiling  wiA  5  miles  and  light  haze  wi A  temperatures  in  the  60s.  PUots  flying  the  saine  route  reported  enroute  weather  as 
occasional  1500  ft  ceilings  wiA  3  miles  visibility  and  scattered  showers.  The  College  Airpoit  is  clear  wiA  bright  sunshine.  Forty- 
five  miles  fiom  the  Ctdlege  Airport  you  have  descended  to  1000  feet  staying  just  below  the  ceilings  and  encoimter  rain  dropping 
visibility  to  und*r  3  miifta  The  terrain  is  flat  Carmland  wiA  no  published  obstacles  above  2S0  ft  tall.  You  decide  to: 


Rank  Order  Alternative 

1  2  3  4  a.  Remain  under  the  clouds,  keep  visual  contact  wiA  the  ground  and  scoot  flirough. 

X  2  3  4  b.  Do  a  180  and  return  home. 

1  2  3  4  c.  Divert  to  the  Madison  County  Airpoit  located  at  7  o’dock  50  NM  and  wait  for  the  worst  weather  to  pass. 

1  2  3  4  d.  Put  it  to  a  vote. 


Airport 


College  Airport 


Madison  County  Airport 


Runway 


5000x100 


4099x100 


3800x75 


Telephone 

Available 

Maintenance 

Yes 

24  his 

Yes 

None 

47e  You  arc  halfway  in  a  two  hour  late  evening  flight  from  the  Regional  Airport  cruising  at  4500  feet  over  a  route  with  an  ME  A  of 
1500  feet.  The  weather  has  been  clear  as  forecast  when  without  ajsy  warning  you  find  yourself  in  a  cloud.  You  decide  to: 


Rank  Order  Alternative 

1  2  3  4  a.  Continue  straight  ahead  for  a  while  and  sec  what  ha^jpens. 

1  2  3  4  b.  Make  a  180  degree  level  turn  and  get  out. 

1  2  3  4  c.  Start  a  wings  level  shallow  descent  to  gel  under  it. 

1  2  3  4  d.  Start  a  wings  level  climb  to  get  on  top. 

48.  You  aie  packing  your  flight  kit  to  go  on  a  VFR  cross  country  trip  borne  for  the  Christmas  Holidays.  In  addition  to  the 
sectLonal  and  flight  plan,  you  usually  include  current  editions  of: 

Rank  Order  Alternative 

1  2  3  4  a.  Take  a  full  set  of  IFR  charts  and  tcnninal  plates  for  the  section  of  the  country  in  which  you  fly. 

1  2  3  4  h.  Take  only  the  VFR  sectional  and  flight  plan. 

1  2  3  4  c.  Plot  what  IFR  information  you  think  will  be  helpfiil  on  the  sectional, 

1  2  3  4  d.  Always  cany  a  full  set  ofIFR  charts  and  plates  on  a  cross  country. 

49.  When  you  get  your  weather  briefing  for  a  cross  country  flight  requiring  at  least  one  fuel  stop,  which  part  of  the  forecast  do  you 
consider  the  most  critical: 

Rank  Order  Alternative 

1  2  3  4  a.  The  weather  at  the  departure  point 

1  2  3  4  b.  En  route  weather  to  the  fuel  stop. 

1  2  3  4  c.  The  weather  at  the  fuel  stop. 

1  2  3  4  d.  Weather  at  the  final  destination. 
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50.  Tike  enroute  weather  briefing  for  the  three  hour  cross  (xiuntiy  was  for  scaitered  thunderstorms  along 

enough  there  is  a  cluster  of  cells  developing  dead  ahead  on  your  route  of  flight  Other  dusters  have  sprung  up  on  each  ride  of  you, 

and  behind  all  dose  to  20  miles  away.  You  decide  to: 

Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Proceed  looking  for  a  route  around  or  through  the  rain  shafts  which  will  allow  you  to  remain  VFR. 

b.  Fly  iqiwind  of  any  cloud  build  ups  and  stay  VF3L 

c.  If  the  downwind  route  around  dark  cdls  is  the  only  dear  way,  keqk  at  least  20  miles  from  the  closest  cell, 
d  Find  an  airport  bdow  in  VMC,  land,  and  wait  until  the  thunderstorms  pass  and  the  nmte  is  clear. 


51.  It  had  rained  all  day,  but  the  front  pushed  south  ofyou  and  cleared  the  sides.  You  are  out  with  two  friends  on  a  sight  seeing  trip 
to  the  hills  40  miles  away  and  plan  to  be  back  before  dark.  With  sunset  still  an  hour  away  you  notice  ground  fog  beginning  to  form. 
You  dedde  to: 

Rank  Order  Alternative 


12  3  4 
12  3  4 
12  3  4 
12  3  4 


a.  Apply  full  power  and  race  back  to  the  home  airport 

b.  Flight  Watch  and  cruise  back  home. 

c.  Can  on  your  home  airfield's  CATF  to  see  if  anyone  is  there  and  can  tell  you  what  the  weather  is  doing. 

d.  Go  directly  to  an  airport  you  know  is  closer  tlm  your  home  airport  land  and  find  out  what  the  weather 
is  doing. 


THANK  YOU! 

PLEASE  CHECK  THAT  YOU  HAVE  RESPONDED  TO  ALL  THE  SCENARIOS,  WE  FIND  THAT  SOMETIMES  THE 

PAGES  STICK  TOGETHER. 

MAKE  SURE  YOUFILL  OUT  THE  PILOT  INFORMATION  FORM. 

If  you  want  infonnation  on  the  results  of  this  study,  please  enclose  a  note  with  your  name  and  address  (don’t  put  them  on  your 

answer  sheets),  and  we  will  send  you  a  copy  of  the  report 
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APPENDIX  C 


GENERAL  AVIATION  PILOT  INFORMATION 


Demographic  Information 


CERTIFICATES  HELD  N 

Private  Pilot . 242 

Commercial  Pilot . 0 

Airline  Pilot  Transport . 4 

Certified  Flight  Instructor . 0 

Certified  Flight  Instructor  Instrument . 0 


PERCENTAGE  (%) 

. 98 

. 0 

. 2 

. 0 

. 0 


RATING 

Instrument . 73 

Single-Engine . 243 

Multi-Engine . 26 


30 

99 

11 


STATE  OF  RESIDENCE 


Arkansas . 4 

Arizona . 2 

California . 1 

Florida . . 1 

Montana . 1 

North  Carolina . 1 

New  York . 63 

Ohio . 7 

Oregon .  1 

Tennessee . 1 

Texas . . 129 

Washington . 34 


..2 

..1 

<1 

<1 

<1 

<1 

26 

..3 

<1 

<1 

53 

14 


AGE 

10-19 

20-29 

30-39 

40-49 

50-59 

60-69 

70-79 

80-89 


3 . 1 


21 . 

. 9 

42 . 

. ...17 

79 . 

. 32 

54 . 

. . . 22 

36 . 

. 15 

,.8 . 

. 3 

1 . . 

. <1 

SEX 

Male . . . . . 235 

Female . 9 


MILITARY/PILOT  CIASSIFICATION 


Yes . 

Class  I 
Class  II . 


96 

..4 


..7 

13 

40 


..3 

15 

17 


Flying  Hours 


TOTAL  HOURS  PERCENTAGE  (%) 

1-250 . 36 

251-500 . 30 

501-750 .  10 

751-1000 . 7 

1001-1250 . 5 

1251-1500 . 2 

1501-1750 . 2 

1751-2000 . 4 

2001-2250 . 1 

2251-2500 . 1 

2501-2700 . 1 

2701-3000 . 0 

3001-3250 . 0 

3251-3500 . 1 

TOTAL  WEATHER  HOURS 

0-49 . . . 

50-99 . 9 

100-149 . 4 

150-199 . 3 

200-249 . 1 

250-299 . 1 

300-349 . 2 

Over  350 . 0 

HOURS  OF  INSTRUCTION 

0-9 . 2 

10-19 . 0 

20-29 . 1 

30-39 . 8 

40-49 . 14 

50-59 . 10 

60-69 . 13 

70-79 . 12 

80-89 . 6 

90-99 . 4 

100-109 . 7 

110-119 . 6 

120-129 . 4 

130-139 . 2 

140-149 . 2 

150-159 . 5 

160-169 . 0 

170-179 . 1 

180-189 . 2 


TOTAL  NIGHT  FLYING  HOURS 


0-24 . 56 

25-49 . 15 

50-74 . 13 

75-99 . 3 

100-124  4 

125-149 . 1 

150-174 . 2 

175-199 . -2 

200-224 . 2 

Over  225 . 1 

FLYING  HOURS  LAST  60  DAYS 

0-4 . -  40 

4- 9 . 14 

10-14 . 15 

15-19 . 10 

20-24 . 8 

25-29 . 6 

30-34 . 3 

35-39 . 1 

40-44 . : . 3 

DAY  HOURS  LAST  60  DAYS 

0-4 . 39 

5- 9 . 18 

10-14 . 14 

15-19 . 10 

20-24 . 6 

25-29 . 6 

30-34 . 3 

35-39 . 2 

40-44 . 1 

WEATHER  LAST  60  DAYS 

0 . 78 

1 . 6 

2 . 2 

3  . 4 

4  . 3 

5  . 1 

6  . . . 2 

Over  6 . 2 

180-189 . 2 

NIGHT  LAST  60  DAYS 

0-4 . 91 

5-9 . 6 

10-14 . 1 

15-19 . 0 

20-24 . 1 
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Years  as  a  Pilot 


INSTRUCTION  LAST  60  DAYS 

0-4 . 92 

5-9 . : . 3 

10-14 . 3 

15-19 . 0 

20-24 . 1 

A/C  Most  Frequently  Flown 

NUMBER  OF  ENGINES  PERCENTAGE  (%) 

1  . 93 

2  . 6 

IFR  CAPABLE? 

Yes . 75 

No . 25 

AUTO  PILOT? 

Yes . 37 

No . 63 

WEATHER  RADAR? 

Yes . 9 

No . 91 

TRANSPONDER 

Yes.... . 96 

No . . . 4 

OWN  AIRCRAFT? 

Yes . 50 

No . 50 

HOURS  CRUISE  CAPABILITY 
2 . 1 

3  . 11 

4  . 36 

5  . 35 

6  . 12 

7 . 3 


NUMBER  OF  YEARS  PERCENTAGE  (%) 


0-4 . 24 

5-9 . 26 

10-14 . 16 

15-19 
20-24 
25-29 
30-34 
35-39 


Over  40 
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APPENDIX  D 


SUMMARY  STATISTICS  BY  SCENARIO 


.Expgn.Hank 

Risk  -  1 

Risk -2 

Risk  -  3 

Risk -4 

1 

b  d  c  a 

12.61 

38.77 

40.97 

54.64 

2 

b  dc  a 

9.90 

40.61 

54.19 

60.97 

3 

d  b  c  a 

5.68 

39.19 

44.84 

60.48 

4 

c  b  d  a 

30.71 

45.00 

61.93 

84.33 

5 

d  b  c  a 

10.55 

15.06 

17.73 

51.68 

6 

bead 

17.13 

17.83 

20.70 

52.83 

7 

cab  d 

40.33 

50.83 

53.67 

56.67 

8 

a  d  c  b 

16.28 

23.59 

24.90 

27.14 

9 

db  c  a 

4.97 

36.21 

59.48 

64.48 

10 

deb  a 

16.22 

23.89 

36.96 

47.50 

11 

cad  b 

12.00 

41.67 

43.90 

61.83 

12 

c  b  d  a 

9.45 

44.68 

47.74 

63.87 

13 

dbc  a 

7.26 

23.39 

39.52 

63.23 

14 

b  da  c 

29.84 

45.97 

59.03 

70.32 

15 

edab 

8.33 

15.20 

34.87 

56.33 

16 

a  d  b  c 

22.63 

50.26 

54.74 

59.74 

17 

beda 

12.06 

18.23 

39.12 

40.88 

18 

abed 

6.05 

48.58 

54.47 

67.89 

19 

d  b  c  a 

14.72 

20.00 

25.83 

47.22 

20 

deb  a 

11.05 

25.79 

48.42 

76.32 

21 

b  acd 

13.06 

36.94 

40.83 

41.84 

22 

d  ab  c 

8.06 

24.72 

41.67 

54.72 

23 

bacd 

17.78 

46.94 

52.17 

62.35 

24 

bade 

11.95 

13.68 

38.95 

47.37 

25 

b  dac 

15.53 

50.83 

61.05 

74.05 

26 

dbac 

20.00 

26.39 

55.28 

61.11 

27 

c  b  d  a 

14.72 

33.06 

46.94 

66.67 

28 

beda 

27.63 

32.37 

42.89 

56.58 

29 

edab 

13.44 

14.22 

24.78 

28.23 

30 

abed 

21.67 

25.00 

49.17 

61.39 

31 

dcab 

13.64 

33.06 

38.06 

53.06 

32 

b  d  ac 

5.79 

26.58 

67.10 

86.95 

33 

adbe 

15.28 

30.00 

37.50 

57.50 

34 

d  c  b  a 

7.83 

16.25 

21.42 

43.50 

35 

beda 

2.25 

21.67 

45.83 

64.17 

36 

deb  a 

16.42 

30.00 

34.58 

39.83 

37 

b  de  a 

4.58 

17.08 

47.67 

52.50 

38 

b  d  c  a 

17.92 

18.33 

42.08 

58.33 

39 

bacd 

25.45 

26.36 

30.00 

38.64 

40 

dacb 

22.92 

30.75 

36.25 

40.50 

41 

bade 

9.83 

23.33 

31.25 

36.67 

42 

bacd 

17.50 

18.33 

48.75 

63.33 

43 

edab 

10.83 

38.33 

65.00 

75.83 

44 

c  d  b  a 

2.08 

38.33 

51.25 

79.17 

45 

bade 

13.27 

23.64 

23.64 

23.73 

46 

bead 

17.50 

33.75 

76.67 

79.54 

47 

d  b  c  a 

32.92 

50.17 

50.42 

58.33 

48 

bead 

16.45 

28.18 

30.45 

33.18 

49 

ebad 

32.73 

33.64 

35.00 

42.54 

50 

dbca 

18.64 

44.54 

51.82 

64.09 

51 

d  b  c  a 

13.18 

39.54 

40.00 

42.73 

Alternatives  are  listed  in  the  order  given  by  the  expert  panel  —  the  first  alternative  listed  was  ranked  #1  by  the 
expert  panel,  and  so  on. 

Risk  values  are  those  assigned  by  the  expert  panel  to  the  four  alternatives.  The  Risk-1  value  is  that  gs<igr».fl  to 
the  alternative  tanked  #1  by  the  expert  panel,  and  so  on. 
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APPENDIX  E 


SOFTWARE  AND  ANALYSIS 

Data  analysis  for  this  project  was  accomplished  on  a  Unisys  1 100  series  mainframe  computer  at  the  Human 
Resources  Directorate  of  the  USAF  Armstrong  Laboratory  (AL/HR)  and  on  the  IBM  RS-6000  RISC  model  530 
machine  at  the  Metrica  facility.  All  software  used  was  developed  by  and  is  the  property  of  the  USAF  Armstrong 
Laboratory,  Brooks  AFB,  San  Antonio,  Texas.  More  specifically,  in  addition  to  utility  and  custom  software  required 
to  reformat  input  files,  data  analysis  was  carried  out  using  software  packages  described  in  the  Mathematical  and 
Statistical  Library  of  the  Armstrong  Laboratory.  This  library  is  reviewed  in  Albert,  W.  G.  &  Whitehead,  L.  K., 
MATHEMATICAL  AND  STATISTICAL  SOFTWARE  INDEX:  SECOND  F.DTTTON  (AFHRL-TP-85-47, 
August  1986).  This  index  provides  a  brief  description  of  each  package  and  external  references  when  available. 
Detailed  program  documentation  is  only  available  on  the  Unisys  under  their  “@DA*DA.ADOC”  retrieval  system. 

The  Unisys  1 100  is  scheduled  for  shut-down  and  replacement  in  October  1997  by  an  IBM  RISC  machine  at  AL/ 
HR  which  is  already  in  operation,  but  not  yet  accessible  to  outside  users.  It  is  unknown  to  this  contractor  as  to  which 
Unisys-resident  programs  are  being  converted  for  operation  on  the  RISC  platform  and  which  will  be  “lost”.  Metrica 
has  already  produced  the  RISC-based  version  of  ASCII  CODAP. 

This  project  used  the  GRPREL  (Group  Reliability,  Staley  &  Weissmuller,  1981)  analysis  procedure.  A  brief 
description  of  this  package  is  provided  below.  Additional  information  of  the  use  of  this  analytic  tool  is  available  in: 
Albert,  etal,  1 994;  Christal& Weissmuller,  1976, 1988;  Goody,  1 976;  Phalen& Mitchell,  1993;  Staley& Weissmuller, 
1981;  and,  Weissmuller,  Phalen,  &  Tartell  (1997). 

GRPREL 

GRPREL  is  the  standard  interrater  reliability  in  the  CODAP  system.  For  a  given  list  of  items  rated  by  a  set  of  Subject 
Matter  Experts  (SMEs),  this  program  reports  two  measures  of  interrater  agreement  (R  and  R  ).  The  R  value 
indicates  the  reliabilityofthe  observed  setofratings — while  0. 1 0  is  consideredaminimum  Vor  usable  rater  agreement, 
a  value  of  0.20  or  greater  is  desired.  The  R  value  is  driven  by  the  number  of  raters  actually  used.  Although  an  R 
of  0.90  is  usually  desired,  it  may  not  be  practical  in  a  particular  study  because  of  a  small  number  of  raters  (SME^ 
that  may  be  subdivided  even  further  into  smaller  groups  based  on  policy  differences.  The  GRPREL  program  also 
computes  means  and  standard  deviations  for  each  item  in  the  list.  Item-level  reports  are  printed  in  three  orders:  original 
sequence,  ordered  descending  y-mean  value,  and  ordered  descending  on  standard  deviation.  GRPREL  computes 
each  rater’s  correlation  with  the  full-group  mean  vector  and  uses  a  probability  evaluation  to  recommend  the  removal 
of  deviant  (non-cooperative  or  reversed  scale)  raters.  The  program  can  automatically  iterate  and  remove  flagged 
raters  until  either  a  sufficienct  level  of  agreement  (R  =  0.90)  is  reached  or  no  raters  can  be  found  with  a  probability 
(of  deviant  rating)  above  0.95 . 
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